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1h INTRODUCTION 


A. BACKGROUND 

On October 5, 1990, the Navy International Programs Office 
(IPO) instituted a five-step evaluation method using a table 
of 14 success criteria, grouped in three sections, to evaluate 
Research and Development (R&D) programs proposed as candidates 
for international codevelopment agreements between the U. S. 
Government and allied nations. Before the development of the 
success criteria, candidate programs were qualitatively 
evaluated using a lengthy worksheet. With the introduction of 
the success criteria, candidate programs were evaluated to 
identify those programs with a low likelihood of success in 
the international arena, and so direct the IPO’s resources at 
those candidate programs more likely to succeed. 

The evaluation consisted of applying the 14 success 
criteria twice to each candidate program, once to rate U. S. 
participation in the program and once to rate allied 
participation. Six strength ratings were then calculated, one 
For each section of criteria for both the U. S. Government’s 
and the foreign participation. These paired ratings were 
compared to each other; based on their "degree of 
convergence," a weighted factor ranging from 0.5 to 1.5 was 


then applied to each paired ratings. The three resultant 


weighted rates were added to arrive at a final composite 
strength rating for each candidate program. Eighteen 
candidate programs were thus evaluated in the following 
months; they were ranked in descending order according to 
their composite strength ratings. 

On July 30, 1991, IPO revised the success criteria, 
dropping two items, substituting in for two others, and 
regrouping the criteria into four sections vice three. 
Generally, the five-step evaluation process remained 
unchanged. The new procedure did call, however, for only a 
Single "strength matching" weight to be applied to the entire 
evaluation to determine the candidate’s composite strength, 
this instead of the area "degree of convergence" weights. 
Four additional candidate programs and six of the original 18 
candidate programs were evaluated using the revised criteria 
and procedures. The results revealed some anomalies: Noite 
only did the four additional candidate programs score 
generally lower than the six original programs, but the 


priority ranking of some original programs was reversed. 


B. OBJECTIVES 

The general objective of this research was to examine the 
impact that the revised evaluation criteria and procedures had 
on the priority ranking of candidate programs, and so verify 


the accuracy of the revised criteria and procedures at 


indicating a program’s potential for success as_ an 


international codevelopment agreement. 


C. RESEARCH QUESTION 
The primary research question addressed by this research 
is as follows: 
® How can the evaluation conducted by the Navy International 
Programs Office on programs being considered for 
nomination as international codevelopment agreements be 


mMerined SO aS tO OLfer a more accurate "front end" 
analysis of the candidate programs? 


Secondary research questions addressed by this research in 
Support of the primary research question are as follows: 


@® What has been the U. S. Government’s participation in 
international armaments agreements since World War II? 


@® What are the current issues affecting efforts at 
international armaments agreements? 


@® What are Department of Defense and Department of the Navy 
policies and procedures regarding international 
codevelopment agreements? 
D. SCOPE, LIMITATIONS AND ASSUMPTIONS 

1. Scope 

This research examines the evaluations performed by IPO on 
22 candidate programs from October, 1990 through July, 1991. 
Data for this research consist of the original and revised 
success criteria, the 18 evaluations conducted using the 
original criteria and procedures, and the ten evaluations 


conducted using the revised criteria and procedures. 


2. Limitations 

The only significant limitation affecting this research 
was an inability to access the computer program that generated 
the evaluation sheets for the International Programs Office; 
access to the computer was slanied for security reasons. In 
lieu of access, computer printouts representing all the 
evaluations performed by the IPO during the period of the 
research were provided. 

3. Assumptions 

The research assumes that the reader has a basic 


understanding of spreadsheet calculations and construction. 


E. LITERATURE REVIEW AND METHODOLOGY 

A literature review was conducted to determine the history 
of the U. S. Government’s participation in international 
armaments agreements since World War II, and to determine 
current issues affecting international codevelopment efforts. 
A review of relevant Department of Defense instructions and 
Department of the Navy reports was conducted to determine 
official policy and procedures. 

The methodology for this research consisted primarily of 
a personal interview with Mr. Frank D. Kenlon, Director for 
International Programs with the Navy International Programs 
Office in September, 1991. Extensive use is made of a 


spreadsheet analysis of the individual evaluation sheets. 


F. OVERVIEW 

Chapter II, Background, recounts the U. S. Government’s 
involvement in international armaments cooperative agreements 
with its allies from World War II to present. It additionally 
explores current issues in codevelopment and coproduction 
agreements, and examines current and proposed Department of 
Defense (DoD) and Department of the Navy (DON) policy 
regarding international agreements. 

Chapter III, Presentation of Data, displays the data used 
in conducting this research. Tables exhibiting the original 
and revised success criteria, and individual evaluation sheets 
for all program evaluations conducted by the IPO are included. 

Chapter IV, Analysis of Data, reproduces the cell-by-cell 
spreadsheet analysis of the various evaluation sheets, and 
correlates the IPO’s success criteria to stated Office of the 
Secretary of Defense (OSD) international program success 
indicators. 

Chapter V offers the conclusions and recommendations 


developed as a result of the research. 


Il. BACKGROUND 


A. U. S. PARTICIPATION IN MODERN MILITARY ALLIANCES 

The United States’ involvement in its present day military 
alliances is largely a result of the economic and military 
uncertainties that characterized the world in the years 
following World War II. The war levied a terrible toll across 
the Europe and Asia, so that 


the global balance of power after the war would be 
totally different from that preceding it. Former Great 
Powers--France, Italy--were already eclipsed. The German 
bid for mastery in Europe was collapsing, as was Japan’s 
bid in the Far East and Pacific. Britain .. . was 
fading . . . . Only the United States and the USSR 
counted, so it seemed; and of the two, the American 
"Superpower" was vastly superior (Kennedy, 357). 


Much of the United States’ power and prestige lay in its 


SCOnOMLe emcee 


The country’s GNP measured in constant 1939 dollars rose 


Euem $88.6 bil iaven (1939) to $135 billion 
(1945) . . . . industrial expansion in the United States 
rose at a faster pace .. . than at any period before or 
Since .. . . Among the Great Powers, the United States 


was the only country which became richer--in fact, much 
richer--than poorer because of the war (Kennedy, 357-358). 


By comparison, the former Great Powers (Germany, Japan, Italy, 
France and Great Britain) all suffered greivous economic 
reversals as a result of the war. Germany was divided up into 
four military occupation zones, and later, into two countries; 
its industrial base, devastated by bombing, was dismantled; 


its national income and output dropped over two-thirds between 


1938 and 1946 (Kennedy, 365). Japan was Similarly occupied; 
its national income fell 43 percent from 1936 to 1946 
(Kennedy, 365). Italy’s GNP in 1945 was only 60 percent of 
its 1938 level (Kennedy, 365-366). France had been plundered 
by the Germans for four years and then suffered from the heavy 
fighting during 1944; its 1945 income was only 50 percent of 
its prewar level (Kennedy, 366). Great Britain fared better 
than those countries, but, even so, was suffering economically 
at the end of the war; it had depleted its gold and dollar 
reserves, worn out its domestic industrial capability, and was 
heavily dependent on the United States for much of its arms, 
shipping, food and other supplies (Kennedy, 367). 
The United States’ economic might was projected worldwide 
by its military power. At the end of the war, the U. S. 
military was comprised of some 12.5 million personnel (7.5 
million deployed overseas); its modern weaponry included 1200 
warships, 3000 bombers, and a monopoly of atomic weapons 
(Kennedy, 358). Of the European powers, only Great Britain 
possessed a credible military force at the end of the war, 
with forces deployed to 
. North Africa, Italy, Germany, (and) Southeast 

Asia. Despite heavy losses, the Royal Navy possessed over 

1,000 warships, nearly 3,000 minor war vessels, and nearly 

5,500 landing craft. RAF Bomber Command was the second- 

largest strategic air force (by far) in the world 

(Kennedy, 367). 


If World War II heralded the emergence of the United 


States aS a superpower, the same must also be said of the 


Soviet Union. The increasing Soviet influence can be seen by 
its territorial growth during and after the war. Its European 
boundaries expanded 


: in the north at the expense of Finland, in the 
center at the expense of Poland; and in the south, 


recovering Bessarabia, at the expense of Rumania. The 
Baltic states of Estonia, Latvia, and Lithuania were 
reincorporated into Russia. Part of East Prussia was 
taken, and a slice of eastern Czechoslovakia . . . was 


also thoughtfully added, so that there was direct access 
to Hungary. To the west and southwest of this enhanced 
Russia lay a new cordon sanitaire of satellite states, 
Poland, East Germany, Czechoslovakia, Hungary, Rumania, 
Bulgaria, Yugoslavia and Albania (Kennedy, 361). 

In the Far East, the Soviets occupied Manchuria, North 
Korea and Sakhalin and linked up with the Chinese Communists 
(Kennedy, 367). After the war, Stalin would reduce the size 
of the Soviet Army by two-thirds 


- « toe the still very substantial total of ja5 
divisions, supported by 25,000 front-line tanks and 1970G3 


aircraft. It would still remain... the largest defense 
establishment in the world--a fact justified ... by its 
need to deter future aggressors and .. . to keep control 


of its newly acquired satellites in Europe as well as its 
conquests in the Far East (Kennedy, 363). 


The Soviet military strength belied the country’s economy, 
which was shattered by the war. Estimates put the number of 
Soviet deaths as a result of the war at 20-25 million; in 
addition, livestock, farm equipment, the transportation 
infrastructure, and houSing were likewise decimated by the 
occupying German Army (Kennedy, 362). In short, "the Russia 
of 1945 was a military giant and, at the same time, 


economically poor, deprived and unbalanced" (Kennedy, 362). 


tnemadeological splitting of Europe into the Capitalist 
Western bloc and the Communist Eastern bloc became apparent 
soon after the end of the War. It was noted in 1946 and early 
in 1947 that 


: various plebiscites and regional elections in the 
German zones (of occupation) were showing "the political 
complexion of West Germany . . . beginning to differ 
markedly from that of East Germany"; the steady 
elimination of any non-Communist elements in Poland, 
Bulgaria, and Rumania was mirrored by the internal 
political crisis in France in April, 1947, when the 
Communists were forced to resign from the government. A 
month after, the same happened in Italy. In Yugoslavia, 
Tito’s political domination (in place of the Allied 
wartime agreements about shared power) was interpreted by 
the West as a further step in Moscow’s planned advance 
(Kennedy, 375). 


A two-pronged strategy was developed by the West to 
contain what was perceived as the Soviet threat to Western and 
Southern Europe. The first element of this strategy 


: was to indicate to Moscow those regions of the 
globe which the United States "cannot permit .. . to fall 
into hands hostile to us." Such states would .. . be 
given military support to build up their powers of 
resistance; and a Soviet attack on them would be regarded 
virtually as a casus belli .. . . The (second and) most 
crucial component of any long-term containment policy 
would be a masSive program of U. S. economic aid, to 
permit the rebuilding of the shattered industries, farms 
and cities of Europe and Japan; for that would not only 
make the latter far less likely to be tempted by Communist 
doctrines of class struggle and revolution, it would also 
help to readjust the power balances in America’s favor 
(Kennedy, 376). 


In Europe, the economic aid element of the containment 
Strategy was implemented by the Marshall Plan in 1947; the 


military support element would come two years later when, 


following the Berlin crisis of 1948-49, the North Atlantic 


Treaty Organization (NATO) was created. 


B. U. S. ARMS COOPERATION EFFORTS SINCE WORLD WAR II 

In the first decade following the signing of the NATO 
treaty, the member nations, confronted by the Sputnik launch 
with an eclipsing of Western missile technology, began to 
cooperate on armaments (Taft, 15). An early expression of 
this cooperative trend was 

: the NATO Co-production Program agreed by the 
Alliance in 1957 . . . . Since then there have been 
literally hundreds of bilateral and multilateral European 
and transatlantic cooperative programs--co-development, 
co-production, licensed production projects, MOU’s, dual 
production, industry-to-industry efforts, outright weapons 
sales, one allied nation to another (Taft, 15). 

Three phases of the Government’s arms cooperation efforts 
Since World War II can be identified. The first, lasting from 
the end of the war until the mid-sixties, 

: was characterized by grant aid known as the 
Military Assistance Program (MAP) .. . . the emphasis was 
on the no-cost (to the recipient) transfer of equipment 
directly from U. S. Force’s inventories--initially surplus 
stocks developed as a result of modernization of our own 
forces--or from additional new production of systems being 
produced for U. S. Forces (Cole, F-3). 

Funding for the MAP came entirely from the U. S. Congress; the 
funds were administered by a central office, the Director of 
Military Assistance, that did not concern itself with the 


implementation of the programs by the various acquisition and 


logistics Commands involved (Cole, F-3). 


10. 


These arrangements worked well. Service hardware 
Program Managers had few if any complaints. The equipment 
being furnished was standard U. S. The requirements were 
easily folded into contracts for equipping U. S. forces; 
the funding was U. S. budget authority; and there were no 
issues of R&D recoupment, administrative surcharges, asset 
use charges, agent fees, co-production, offsets, MOUs, 
etc., that became commonplace for providing U. S. arms to 
friends and allies (Cole, F-3). 

The second phase of arms cooperation overlaps the first 

but can be characterized by 

. a gradual shift from MAP to sales on a government - 
to-government basis usually referred to as Foreign 
Military Sales (FMS), and to direct commercial sales 
(Cole, F-3). 

The Foreign Military Sales program 

2 initially focused on the stronger economies of 
European NATO .. . with the objective of having allies 
obviously able to pay their own way assume larger shares 
of the common defense burden. 

MAP procedures continued for most other countries 
friendly or allied to the United States and also for some 
European NATO allies (Cole, 13-14). 

Responsibility for FMS was transferred from the Director 
of Military Assistance to the newly established position of 
Director of International Logistics, and later, along with 
responsibility for MAP, to the Director of the Defense 
Security Assistance Agency (DSAA) upon that agency’s 
establishment in 1971 (Cole, F-3 - F-4). Congress began to 
curtail MAP funds, with the result that DSAA attention became 
increasingly focused on FMS transactions; FMS credits became 
the means by which countries progressed from MAP to cash sales 
(Cole, F-4). DSAA (conceived as simply the fund manager and 


resident authority for international arms~ programs), 


ual 


eventually became "firmly astride and enmeshed in the 
acquisition and logistics function” (Cole, Fs4)- In respomee, 
the 
OSD acquisition and logistics staffs, the Military 
Departments and Logistics Commands demanded and received 
a greater voice in international arms matters. By 1976 
program direction had become diffused and controversies 
were common (Cole, F-4). 

A DoD task force, charged by Secretary of Defense Rumsfeld 
with reviewing Security Assistance, recommended on January 14, 
1977 that DSAA should report directly to the Defense 
Acquisition Executive vice the Assistant Secretary of Defense- 
International Security Affairs, but this recommendation was 
never acted upon by the incoming Carter Administration (Cole, 
| cle ae 

The third (and current) phase of the Government’s arms 
cooperation efforts is characterized by declining U. S. arms 
exports and an increase in the number of codevelopment 
agreements (Cole, F-4). An examination of this phase occurs 
in later in this chapter as part of the discussion of 


Rationalization, Standardization and Interoperability (RSI) as 


it applies to NATO. 


Co ADVANTAGES AND DISADVANTAGES OF ARMS COOPERATION 
The most obvious advantage to two or more countries 


entering an arms cooperation agreement is economic. Thomas A. 


eZ 


Callaghan notes: 


For centuries commercial economics and trade have been 
moving towards ever-larger markets, providing economies of 


scale and ever more affordable prices . . . . Defense 
economics has bucked this trend. Here we have ever-larger 
product lines, ever-smaller production runs, small 


national markets, and (consequently) ever-higher prices 
(Callaghan, 61) .- 


It follows, then, that an increase in the market for a 
weapon system through arms cooperation agreements would lead 
to fewer product lines, larger production runs, and lower 
prices. This simple fact was 

: first appreciated within Europe, and more so than 

in the United States. There were good reasons for this, 
not least of which were the budgetary constraints with 
which the European states were faced, the range of 
conventional operational requirements they had to meet 
with considerable fewer resources, and the comparatively 


small production run each could afford on any one program 
(EamMeonas, Ll.) . 


Bitttetc — logic holds true formany garms cooperation 
agreement in that all the participating countries would 
experience cost reductions through 

3 improvements in production arising from the 

learning process, through quantity purchase of materials 
and parts, better use of fixed capital and steady cash 
mows Ednonas, t11).. 

Two political arguments can be made as well in favor of 
arms cooperation agreements. First, a weapon system that can 
be demonstrably shown to have reduced unit costs would be more 


likely to be accepted by the citizens of a country, especially 


when the risks associated with developing and producing the 


IES: 


weapon system could be shared with other nations (Edmonds, 11- 
BA ae Secondly, a country that participates in arms 
cooperation agreements with its allies 
demonstrates a willingness to adjust purely 
national priorities in the interest of alliance cohesion 
and, arguably, a willingness to further the wider 
political objectives of creating interdependencies among 
a allies and .. . establishing an integrated defense 
INdUuStmy= (BCMenaS a2). 

A final advantage to arms cooperation agreements is that 
it 1s militarily more desirable for alliance forces operating 
in conjunction with each other to be equipped with 
Standardized weapons, logistics and communications systems 
(Edmonds, 14). 

Set against these reasons to enter arms cooperation 
agreements are the pressures inherent in a sovereign state to 
maintain a unique domestic arms production capability; it 
would be disadvantageous for a nation to enter such an 
agreement when it is that government’s desire 

to limit reliance upon external sources of 
military equipment, to reduce the burden of imports on 
their balance of trade, and to ensure maximum employment 
for workers in defense and defense-related industries 
(Bameonds; 1) % 

As Shaffer notes, 

cooperation in armaments iS not an area of 
national policy isolated from security policy or from more 
general domestic and foreign policy concerns. As a result 
of this linkage, nations desire armament self-sufficiency, 
to the extent economically and politically feasible, in 


order to avoid dependence on others .. . in formulating 
and implementing policy objectives (Shaffer, 27). 


14 


In addition, compelling reasons of national security could 
often deter one country from entering an arms cooperation 
agreement due to the possibility of sensitive technology 


transfer. 


D. FORCES CONTRIBUTING TO ARMS DIVERSITY 

The earliest attempts at arms cooperation in NATO led to 
a certain homogeneity of weapons among the alliance members in 
that "virtually all military equipment was supplied by the 
United States" (Shaffer, 29). This uniformity, driven by 
necessity as much as the booming postwar American economy, 
soon fragmented. 

Five specific factors have been identified as contributing 
to the push toward NATO weapons diversity. The? fabesiee Of 
these factors is NATO’s "overreliance on nuclear weapons to 
the detriment of conventional forces" (Galan Zz ZO" Sy) ; 
especially in the early years of the NATO alliance: 

Presuming themselves safe from conventional attack by 

the USSR and its allies, (NATO) general purpose forces 
were by and large starved of resources and left to 
deteriorate; the member states were left free to a large 
degree to go their separate, sovereign ways in pursuit of 
national security . . . . The United States strongly 
advocated this reliance on masSive retaliation and did not 
particularly mind the burgeoning proliferation of 
different types of conventional weapons systems (Cannizzo, 
Ew). 
The United States’ support of a strategic defense of NATO 


should be viewed in light of its then vastly superior nuclear 
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arsenal and its later commitment of conventional forces to the 
conflicts in Korea and Vietnam. 

The second factor that advanced weapons diversity in NATO 
is closely nellatedmte Ene trast. 


The United States encouraged the postwar rebuilding of 
European defense industries in order not to overtax the 
"arsenal of democracy" and to help spur reconstruction 
efforts) (Canmazze,725/)- 


A third factor derives from the economic problems 
afflicting the United States beginning in the early sixties 
brought on by an unfavorable balance of trade and Ehe Elowses 
gold passing through U. S. borders (Cannizzo, 57): 


The United States patently became more interested in 
forcing the Europeans to buy U. S. equipment and in 
finding ways to offset the cost of stationing U. S. troops 
abroad than it was in fostering weapons cooperation. Such 
policies constituted a source of irritation for the 
European Allies, which was compounded by the "buy 
American" acts . . . . Not only the United States has 
continuing economic difficulties, but the Europeans have 
as well, which today have helped create a situation more 
compatible with protectionism than international 
cooperation canni Z7z07 oe 


Cannizzo identifies the United States’ involvement in 
Vietnam as the fourth factor lending to NATO weapons 
diversity: 

The diversion of U. S. interests and attention to 
Southeast Asia clearly reduced U. S. participation in NATO 
affairs and procurement policies .. . the United States 
drew on its stocks of equipment in Europe and... . this 
left large gaps which remain unfilled even today 
(@GannlZ Zeus) = 

Finally, Cannizzo points to the NATO experience with arms 


cooperation projects themselves as leading to diversity. 


Citing the consortium created to manage the coproduction of 
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the F-16 as an example, Cannizzo points to a myriad of 
problems: 

: charges of bribery and even hints of murder during 
the bargaining and selection phases; continual cost 
increases; acrimonious debate over unfulfilled promises of 
jobs, income, and other benefits; disagreements over 
third-country sales; and even disgruntlement over the 
plane’s performance (Cannizzo, 57). 

Another example of an unsuccessful NATO joint program was 
the attempt by the United States and West Germany to develop 
the MBT-70 battle tank which "ended in failure after more than 
$400 million had been invested" (Kitfield, "Finally," 77). 

The current NATO peluey Oi "Rationalization, 
Standardization, and Interoperability" (RSI) is the latest 
expression of the alliance’s efforts to promote arms 
cooperation with the hope of realizing 

2 not only the militarily desirable goal of 
Standardizing equipment and harmonizing logistical 
procedures, but also the politically desirable goal of 
alliance solidarity (Feldman, 48). 

E. THE EVOLUTION OF RSI 

RSI is intrinsically tied to the theory of "structural 
disarmament" as formulated by Callaghan. Callaghan states 
that the Western nations are 

experiencing (a) disarmament phenomenon of more 
and more money producing fewer and fewer weapons, less 
"readiness," and even less combat sustainability. The 
cause is structural. No nation--not even the US--provides 


a large enough defense market to develop and produce 
everything itself (Callaghan, 61). 
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This section will examine the concept of RSI, and will trace 
its implementation through laws enacted by the United States 
Congress and policy directed by the President and the 
Department of Defense. 

A definition of terms is in order: 

"Rationalization" is defined as any action which 
increases the effectiveness of NATO by improving the 
command-support structure and consolidating the various 
national logistical systems. It includes standardization, 
specialization, interoperability, cross-servicing, and 
general cooperation of military and non-military matters. 

"Standardization" refers to the achievement of the 
closest practical cooperation among forces, sharing of 
systems designs, agreement on operational, administrative, 
and logistic procedures, and common tactical doctrine and 
organization. 

"Interoperability" consists of making different weapons 
systems work together by using common or compatible 
components, fuel, and ammunition (Feldman, 50). 

As waS previously mentioned, a high level Of 
Standardization was achieved in the early years of NATO by the 
Simple fact that most weapons were provided by the United 
States. Continued economic aid to the European allies by the 
United States led to a rebuilding of European industry so 
that, by the late 1950s, "European industry was sufficiently 
recovered to permit the independent development of exclusively 
European weapons systems" (Feldman, 51). 

The Warsaw Pact modernization of the 1970s and the 
realization that NATO was wasting up to $15 billion a year 


because of weapons diversity led to a renewed call for 


Standardization in conjunction with the rearming of NATO 
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(Feldman, 51). But purely national concerns often overrode 
alliance concerns so that 
NATO leaders found it increasingly difficult to 

convince parliaments to spend large amounts of foreign 
exchange on the purchase of weapons from abroad, because 
of their fear of generating domestic unemployment. 
Therefore, most NATO members began in the 1970s to demand 
economic compensation in the form of "offsets" such as 
coproduction or countertrade as a condition of purchase 
(Feldman, 51). 

In the United States, the Government began to take the 
Mmetative in promoting RSI. An amendment to the Defense 
Appropriation Authorization Act of 1975 called for the 
Secretary of Defense to ascertain the cost of the failure to 
Standardize weapons in NATO and to develop a plan for arms 
cooperation (Feldman, 51). The 1976 Nunn amendment encouraged 
reciprocal procurement of equipment between the U. S. and 
other NATO nations; authorized the SECDEF to waive the "Buy 
American" law in the interest of NATO standardization and 
interoperability; and enabled the establishment of bilateral 
reciprocal procurement MOUs between the U. S. and the European 
nations of NATO (Naval, 25). 

President Ford introduced the two-way street concept to 
the Europeans at the 1976 NATO summit, and advised them that, 
to compete effectively with U. S. defense industry, the 


Buropeans would have to sacrifice some national concerns and 


combine their resources (Feldman, 52). 


aby) 


The Carter administration approached arms cooperation from 
two almost diametrically opposing views. On the one hand, the 
administration 


wanted to greatly decrease the U. S. sales of 
weapons around the wertd, thus avoiding a reputation as 
"Merchants of Death. Correspondence, dubbed the "Leprosy 
Letter," was sent to all U. S. embassies early in the 
Carter administration prohibiting support and assistance 
to U. S. defense industry corporations trying to sell 
overseas. Legislation was enacted by Congress disallowing 
foreign marketing expenses to be charged as a valid cost 
of doing business (Naval, 23). 


On the other hand, the Carter administration encouraged arms 
cooperative agreements with allies; in particular, Under 
Secretary of Defense for Research and Engineering William 
Perry greatly furthered the cause of RSI. Through Perry’s DoD 
Directive 2010.6, a three-pronged approach to RSI _ was 
implemented: 


First, the general and reciprocal procurement Memoranda 
of Understanding (MOUs) .. . were to be used to create a 
"two-way street" in defense trade. Each country would 
agree to remove domestic obstacles to the mutual 
penetration of defense markets. Each country would obtain 
exemptions from the other’s customs, duties, tariffs, and 
protectionist legislation. In addition, each country 
would ideally give equal consideration to all qual ae 
bidders regardless of the firm’s nationality . 

The second prong .. . was called "Dual Production, " ane 
attempted to avoid redundant research and development 
(R&D) expenditures by encouraging nations to ae a 
system already developed by another member 

The third . . . strategy for achieving RSI was ‘called 
the "Family of Weapons" approach . . . . it began 
collaboration at the R&D stage, requiring complete sharing 
of technology and _ know-how. It called for the 
complementary development of weapon systems in a 
particular mission area to be allocated to different 
countries, sequentially produced, and then made available 
to other members for production or co-production (Feldman, 
Sy 
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The first of these strategies, the negotiation of MOUS to 
aid in arms cooperation, received widespread support 
throughout NATO. Currently, formal agreements exist between 
the United States and 19 allied countries regarding arms 
cooperation. "Dual Production" continues to be favored by the 
DoD and the U. S. defense industry because "it approached 
Standardization by relying on competitive R&D followed by 
licensed production or co-production" (Feldman, 52). The 
"Family of Weapons" approach was unpopular in America, on the 
industrial side because it called for relinguishing 
proprietary data (Feldman, 52), and on the military side 
because of high level concern "over the flow of critical 
mee@manology to the Eastern bloc" (Kitfield, "Finally," 77). 

The Reagan administration approached arms cooperation with 
an entirely different attitude. Armaments cooperation 

: was pushed on all fronts. The "Leprosy Letter" 

was repudiated; the first international cooperative 
development program was started (Terminally Guided Warhead 
for the Multiple Launch Rocket System); defense 
cooperation offices were established in most major U. S. 
embassies to promote cooperation; and over thirty new 
cooperative development MOUs were signed (Naval, 24). 

Two other developments in RSI implementation during this 
period are the Nunn amendments to the FY85 and FY86 Defense 
Appropriations Authorization Act. The former professes the 
Government’s support of RSI 

: but intends to achieve it by creating an open, 

competitive market in defense equipment--a trading 


structure that is hoped will accomplish more than the 
"piecemeal" government MOUs of (the) past (Feldman, 53). 
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The latter takes a more proactive role by requiring all new 
weapon development programs be examined for possible 
international cooperation; by providing $100M as "seed money" 
for new, international codevelopment programs; and by 
providing $25M to allow the U. S. to test and evaluate systems 
already developed by Allies prior to beginning domestic 
production of Similar systems (Naval, 26). Congressional 
funding in support of cooperative developments climbed from 
$100M in FY86, to $145M in FY87, to $150M in FY88, to Si52raa 
FY89, before falling to $115M in FY90 (Cole, 61). Al Soma 
1987, the Quayle amendment permitted the DoD to cooperate with 
foreign nations from R&D through procurement (Naval, 25-26). 
President Bush has generally continued the arms 
cooperation policies of his predecessor. The Bush 
Administration has 
directed embassy personnel to increase the level 
of assistance provided to U. S. defense companies, created 
the Center for Defense Trade within the State Department, 
and proposed a "defense GATT" (General Agreement on Trade 
and Tariffs) that would allow free and open trade in arms 
and defense technology within the NATO Alliance, and with 
other U. S. Allies. In March 1991, the Administration 
proposed that the Export-Import Bank guarantee up to $1 
billion in commercial loans to members of NATO, Australia, 
Japan and Israel to purchase defense equipment from U. S. 
contractors (U. S = s@enguess 210 
On April 16, 1990, the President issued his policy on 
offset agreements. In essence, the policy prohibits the 
United States Government from participating in 
. any guarantees of offsets to any allied government 


Or participating industry. This participation goes so far 
as to say that no agency can encourage, enter directly 
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into, or commit U. S. companies to any offset arrangements 
as they relate to any sale of defense articles or services 
to other governments. Additionally no U. S. funds may be 
used to finance offsets ... . The new guidance does not 
prohibit negotiations for offsets by any government with 
the commercial industries involved as long as there is no 
We, Se Government encouragement, Participation “or 
commitment for the offset agreements which might be 
finally realized (McCarroll, 6-7). 
F. IMPACT OF RSI 
Objectively, the impact of the various RSI initiatives can 
best be appreciated by examining the balance in defense trade 
between the United States and the European allies. Most 
analysts agree that in the 1970s, the ratio of defense trade 
between the U. S. and its NATO allies was as much as 10:1 in 
favor of the United States (Costello, TO - Kitfield, 
"Obstacles," 86). Following Perry’s RSI initiatives, this 
ratio was reduced to a level of 4.8:1 in favor of the U. S. in 
FY84 (Roos, 23). The figures for FY86 show a further leveling 
off of the trade ratio, to 1.6:1 in favor of the United States 
(Kitfield, "Obstacles," 86; Roos, 23); this last ratio can be 
interpreted considering Nunn amendment. Other interpretations 
can be offered for the decline--Kitfield points out that the 
FY84 figures are, in part at least, because the European 
nations reduced their defense purchases from the United States 
by 45 percent from the previous year (Kitfield, "Finally," 
76). However, Congressional calculations show that the 


transatlantic defense trade ratio, which in 1981 had reached 


eaeugh ©f 12:1 in favor of the U. S., had only dropped to 
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about 5.2:1 in 1988 (U. S. Congress, 49). In any event, iaie 
overall trend proves that some effort has been expended in 
behalf of RSI. 

Another set of figures that reinforces the declining trade 

ratio balances concerns military contracts: 

The amount of U. S. military contracts awarded abroad 
nearly doubled (during the period FY80 to FY86) from $5.4 
billion to $9 billion .. . . During the same periaoce 
foreign military sales (FMS) by U. S. companies were cut 
in half, from $14.8 billion in 1980 to S57.) billi0n wee 
1986) (Kitfield, "Obstacles," 86). 

G. OBSTACLES ON THE TWO-WAY STREET 

Although RSI is vigorously promoted on both sides of the 
Atlantic, very real obstacles impede optimal progress down the 
NWEWO-Wday sstreeter. 

On this side of the Atlantic, a general problem is that 

there is not a clear, national consensus in support of RSI: 

RSI never became a "national decision," its proponents 
having failed to win the blessing of Congress, the 
military services, industry, and labor... . Each group, 
for a variety of reasons, has at one time or another 
obstructed or resisted RSI (Feldman, 53). 

Four major obstacles can be identified: 
® the "not invented here" syndrome; 
® economic protectionism; 


® technological isolationism; 


@® the NATO burdensharing controversy (Feldman, 54). 
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The "not invented here" syndrome is primarily a military 
Criticism of RSI which rests on three pillars. First, the 
Pentagon is rightfully concerned about any arms cooperative 
program that could "leave the U. S. dependent on a foreign 
source for any component of its defense needs" (Feldman, 54). 
Secondly, the Pentagon reasons that "a variety of non- 
standardized defense systems would be more likely to thwart 
Soviet countermeasures than a few standardized systems" 
(Feldman, 54). Finally, some officials fear that the quality 
of our forces’ weapons might be sacrificed to achieve the 
political goal of armaments cooperation (Feldman, 54). 

The obstacle of economic protectionism is a Congressional 
reaction to RSI. Although the MOUS were negotiated to ease 
the restrictions of protectionist statutes, elected officials 
can still be influenced by both industry and labor in their 
districts. For instance, the 


Aerospace Industries Association (AIA) and the 


Electronic Industries Association (EIA) . . . have 
testified before Congress . . . that RSI is a form of 
"affirmative action" for the Europeans ... . The 


implications for U. S. jobs are great, according to the 
AIA, which estimates that for every $1 billion in export 
Sales lost, 70,000 U. S. jobs disappear as well (Feldman, 
55). 
Constraining armaments cooperation, Congress has supported the 
Arms Export Control and the Export Administration Acts; has 


enacted the requirement that the Department of Commerce 


participate in the already slow MOU process; and has included 


ZS 


individual protectionist measures in most of the recent 
Defense Authorization Acts (Naval, 26). 

Perhaps the greatest obstacle on the two-way street is the 
issue of burdensharing, which arises from the perception that 
"Some countries (in the NATO alliance) may be doing more, or 
not doing enough, in contributing to collective secu 
(Lightburn, 26). The issue has often been couched in terms of 
each member’s financial and military contribution to NATO 
expressed as a percentage of GNP/GDP. A recently completed 
study by NATO has explored the burdensharing problem. 

The study, done in 1988, recognized that there were many 
factors, besides financial and military, that comprised a 
country’s contributions to the alliance, and that these 
factors could not all be quantified or compared (Lightburn, 
2 Gime Nevertheless, an assessment for each country’s total 
contribution was estimated and agreed upon by all other 
countries in the alliance (Lightburn, 27). As a result of 
these assessments, proposals were formulated to maintain the 
quality of the allied defence, to redress some deficiencies 
that the assessments discovered, and to enhance security in 
the future (Lightburn, 27-28): 

In the first of these categories, .. . countries were 
asked to improve their performance in meeting NATO goals, 
to enhance the forces which are committed to the Alliance, 
to improve force sustainability, to continue to support 
commonly funded programmes, to strive for improved 
armaments cooperation, to provide more assistance to 
developing defence industry nations and to search for 


greater efficiencies in the defence area. In the second 
category, . .  . countries were asked to consider 
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improvements to the NATO Composite Force for North Norway, 
to the Allied Command Europe (ACE) Mobile Force and to the 
on-call naval force in the Mediterranean, to improve where 
possible host country support arrangements, to enhance 
reinforcement capabilities and to improve the Alliance’s 
airborne early warning capability. The third category 
challenged countries to look at a better rationalization 
of existing defence arrangements, to examine and improve 
the potential of reserve and mobilizable capabilities, and 
to explore other ways of enhancing security through 
additional common or joint funding of requirements 
efigm@eburn, 28). 


H. DOD SUPPORT OF ARMAMENTS COOPERATIONS 

The Department of Defense has supported both Executive and 
Congressional efforts to further armaments cooperations. 
Directives are in place which form the basis of Defense 


Department policy on international armaments cooperation. 


Some key directives are listed below: 


® DoD Directive 2000.9, eine Gictedolc mec Produlet lon 
Projects and Agreements Between the United States and 
Other Countries or International Organizations," dated 


January 23, 1974. 


@® DoD Directive 3100.3, "Cooperation with Allies in Research 
and Development of Defense Equipment," dated September 27, 
mG 3. 


@® DoD Directive 3100.4, "Harmonization of Qualitative 
Requirements for Defense Equipment of the United States 
and Allies," dated September 27, 1963. 


@® DoD Directive 5530.3, "International Agreements," dated 
June 11, 1987. 


Directive 2000.9 is currently being revised; its revision 


would cancel the original 2000.9 and Directives 3100.3 and 
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3100.4, along with various memoranda issued by the Department. 
The draft revision proposes a ten-point DoD policy concerning 
international armaments cooperation; it says that it would be 
DoD policy that 
® The DoD would cooperate with U. S. Allies in the research 
and development, testing and evaluation (RDT&E) ; 
production; procurement; sale and follow-on support of 


conventional defense equipment when such cooperation 
promotes U. S. foreign policy objectives; U. S. security 


interests; the fulfillment of validated operational 
requirements; an approach for integrating DoD’s 
requirement generation, acquisition management, and 


planning, programming and budgeting systems; and the 
maintenance or improvement of the U. S. defense industrial 
base. 


® The DoD would participate in cooperative programs that 
foster the collective security by encouraging participants 
to adopt interoperable equipment, compatible support 
arrangements, and common operational doctrine; by 
encouraging participants to invest appropriately in modern 
conventional defense; by making the most efficient use of 
the total scientific, technical, financhalsandeandusemaaet 
resources available to participants; and by fostering 
defense industrial capabilities in all participating 
countries to meet the military requirements of the 
alliance. 


@® The DoD would maximize its resources available for defense 
of the alliance by conserving DoD resources through access 
to those foreign defense goods and services that provide 
cost-effective alternatives for meeting U. S. operational 
requirements; and by facilitating the common defense of 
the alliance through promoting equitable access to the 
most cost effective goods, technology and services, 
reducing duplicate efforts at RDT&E, production and 
Support, and by fostering a more efficient market for 
defense goods. 


® The DoD would encourage consensus and validation of 
military requirements through treaties or agreements at 
the earliest practicable stage in the requirements’ 
generation, and through all stages of the requirements’ 
POramnul alt Lom, 


® The DoD would authorize the release of classified or 
unclassified U. S. information and technology if such 
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disclosure was subject to a Technology Assessment Control 
Plan and if such disclosure complied with law and other 
DoD Directives. 


® The DoD would protect and advance the U. S. industrial 
base in negotiating and implementing cooperative 
agreements by considering the effects of the agreements on 
the defense industrial base; and by interfacing with the 
Department of Commerce (DOC) regarding the implications of 
agreements on the industrial competitive position of U. S. 
PMU S aay. 


® The DoD would prepare an industrial base factors analysis 
for each agreement to assess the defense industrial 
implications of cooperative agreements. 

® The DoD would not encourage, enter directly into, or 
commit U. S. firms to any offset arrangement through a 
cooperative agreement except as directed by the President 
through the National Security Council. 

® The DoD would maintain a consistent approach to 
cooperative agreements by negotiating and concluding all 
agreements in accordance with DoD Directive 5530.3 or the 
Security Assistance Management Manual, as applicable. 

® The DoD would continually monitor and review cooperative 
programs to ensure consistency with U. S. interests and 


objectives, and would maintain a consolidated index of all 
cooperative agreements (Draft, 2-6). 


DoD Directive S5310).3 "Tnternational Agreements" 
establishes the requirements for, and restrictions. on, 
authority to negotiate or conclude an international agreement. 
The directive defines "international agreement" broadly as any 
agreement concluded with one or more foreign governments that 
iS signed or agreed to by personnel of the U. S. Government, 
that signifies the intention of the parties to be bound in 
iMrzecnmational law, and that is identified as an international 


agreement or other similar language (2-1). The definition 
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specifically excludes certain other official compacts made by 
Government personnel such as contracts made under the Federal 
Acquisition Regulations (FAR) and Foreign Military Sales (FMS) 
Credit Agreements (2-1). DoD Directive 5530.3 establishes 
Standard procedures for requesting authority to negotiate or 
conclude international agreements, and assigns responsibility 
for that authority to the Under Secretary of Defense for 
Policy (USD(P)). For certain categories of agreements, the 
directive delegates authority to negotiate and conclude 
international agreements to the Service Secretaries, the 
Chairman of the Joint Chiefs of Staff (CJCS), the Directomee: 
the Defense Security Assistance Agency (DSAA), the Under 


Secretary of Defense for Acquisition (USD(A)), and others. 


I. DON SUPPORT OF ARMAMENTS COOPERATION 

In December, 1989, a Naval Research Advisory Committee 
(NRAC) Report on International Research and Development (NRAC 
Report No. 89-7) was completed. The committee panel was 
comprised of representatives from Government, private 
industry, and universities, and chaired by Mr. Gerald Cann, 
then of General Dynamics Corporation. The panel had been 
formed to address the following questions as they pertained to 
the Navy’s international cooperative research and development 
(IR&D) programs: 


@® How may the Navy enhance identification and assessment of 
foreign technologies to facilitate both the rapid and long 
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term incorporation of these technologies into naval weapon 
systems? 


® How may the Navy better identify potential cooperative 
development programs? What fundamental elements impact 
the success or failure of a cooperative development? 
Which elements should be considered in determining 
preterity ? 


@® What role can industry assume to facilitate international 
cooperative programs and take advantage of available 
foreign technologies? What 1S necessary to maximize 
industry involvement in these programs and strengthen the 
U. S. industrial base? How may IR&D funding be used in 
these programs? 


® What incentives should be offered to promote international 
cooperative programs? How may the Navy reward program 
managers and industry for cost savings achieved through 
successful programs? 


® How may international programs be used to promote the 
maritime strategy objectives of the Navy? 


® How may the coordination and review of Navy international 
programs be improved? What organizational changes are 
necessary? 

@® Are the fiscal goals established by the OSD for service 
participation in international programs, practical and 
adequate? 

@® How will new legislation, requiring Department of Commerce 
review of all proposed international cooperative 


development Memoranda of Understanding (MOUs), impact Navy 
programs and U. S. industry (Naval, 15-16). 


An immediate cause for the panel’s formation was the fact 
that two ongoing cooperative programs had been recently 
cancelled and several others appeared to be in trouble (Naval, 
a). Factors underlying the cancellations were readily 
discernible. These included changes in the external 


environment in which IR&D is conducted; the lack of a formal 
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national policy on international armaments cooperation; and 
the means by which armamentS cooperative agreements are 
impacted by actions of Congress, the Department of Defense and 
the Department of the Navy (Naval, 3). 

The panel recognized several trends in the external 
environment that impact on the Navy’s IR&D efforts, central of 


which is the increasing "globalization of the economy, 


industry and trade. This, in turn, has fostered increasing 
interdependence among nations" (Naval, 21). The panel 


acknowledged an erosion in the United States’ traditional 


ae leadership position in several important 
(technological) areas, Such as microelectronics, ceramics 
and manufacturing process technology. Foreign capability 
in the design of integrated circuits, launch vehicles and 
commercial aircraft underscores the general trend toward 
technology leveling. As a result, intense competitive 
pressure has promoted widespread dual use of our most 
advanced technologies (Naval, 21). 


A final environmental trend identified by the panel was 
the fact that our focus on the threats facing our nation was 
shifting from a military threat by the Soviet bloc to an 
economic threat by the other industrialized nations of the 
world, specifically the Pacific Rim nations and Western Europe 
(Naval, 21). The panel attributed industry’s narrow 

: . preoccupation with short term profit and corporate 

book value (with putting) us at a distinct disadvantage in 


dealing with Japan’s longer goal of securing market share 
(Nasal 8 2s 





Concerning the Government’s national policy on 


international armaments cooperation, the panel was quick to 
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point out that the Government has no formal national policy 
even though 

, armaments cooperation with North Atlantic Treaty 

Organization (NATO) and other friendly countries has been 
an integral part of DoD’s RDT&E and production procurement 
programs since World War II (Naval, 23). 
Instead of a formal, written policy on international armaments 
cooperation, an ad hoc policy evolved "in accordance with the 
prevailing international views held by each administration" 
(Naval, 23). 

The panel recognized that the Department of the Navy has 
generally followed OSD direction for international armaments 
cooperation by issuing SECNAV instructions that direct 
compliance with the relevant DoD Directives and by generating 
its own set of OPNAV instructions to provide guidelines for 
conducting MOU negotiations (Naval, 29). The NRAC panel did 
level some criticism at DoN for not being sufficiently 
sensitive to U. S. industrial base concerns and for the ultra- 
conservative position the Navy has traditionally taken 


regarding the licensing of its equipment for use and sales 


overseas (Naval, 30). 


J. THE NAVY INTERNATIONAL PROGRAMS OFFICE (IPO) 
Coordination of the Navy’s international armaments 

cooperative efforts is accomplished by the International 

Programs Office (IPO), reporting to the Deputy Assistant 


secretary of the Navy for International Policy (DASN(IP)). In 
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IPO, the Director for International Agreements is responsible 
for oversight of the acquisition MOUs, for assisting in the 
development and negotiations of MOUs by the Program Managers, 
and for poliey cocrdinaeron- 

The IPO office traditionally, at regular intervals, issued 
a "call" for Candidate Nomination Proposals (CNPs) throughout 
the various Program Offices of the Navy. Offices whose 
programs were fully funded typically declined to submit CNPs 
while offices facing shortfalls and budget cuts proposed 
programs for MOU consideration. 

Recent procedures established by DoD Instruction 5000.2, 
"Defense Acquisition Management Policies and Procedures," may 
lead to an increase in both the quantity and quality of 
candidate programs. The instruction requires that a report be 
submitted to Congress annually regarding international 
cooperative efforts. The report, the Cooperative Research and 
Development Projects Report, is prepared by the Deputy Under 
Secretary of Defense for International Programs (DUSD(IP)), 
and reviews Acquisition Category I (ACAT I), Major Defense 
Acquisition Programs, activity for the year (11-D-1-3). The 
report will provide Congress with two pieces of information: 

® A description of status, funding and schedule of existing 

cooperative research and development projects for which a 

Memorandum of Understanding (or other formal agreement) 

has been entered into; 

® A description of the purpose, funding and schedule of any 
proposed new projects included in the President’s budget 


proposal for which a Memorandum of Understanding (or other 
formal agreement) has not been entered into (11-D-1-3). 
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The instruction also requires that a Cooperative Opportunities 
Document be included as Annex G to the Integrated Program 
Summary document as part of the Defense Acquisition Board 
(DAB) Milestones Review procedures (11-C-3). This requirement 
applies to all Acquisition Categories programs at Milestones 
I through IV; the Cooperative Opportunities Document would 
address the following areas: 
® Are there any Similar projects in development or 
production by one or more major allies of the United 
Dea res 
® If yes, could that project satisfy, or be modified in 
scope so as to satisfy, the military requirements of the 
United States? 
® What are the advantages and disadvantages of seeking to 
structure a cooperative development program with one or 


more Allied nations? 


® What alternate forms of cooperation could be appropriate 
for this project (DoD Manual 5000.2M, 4-H-1-1 and -2)? 


The IP Office also targets likely MOU prospects through the 
annual Planning, Programming and Budgeting System (PPBS) 
review process, during which the office focuses primarily on 
mid-level programs (ACAT II programs, the lower range of ACAT 
I programs, and the upper range of ACAT III programs). 

Once the IP Office identifies a candidate cooperative 
program, either through a successful PPBS targeting or a call 
for candidate programs, the office discusses the program with 
the originator to assess the program’s potential as a 


cooperative agreement. For programs with significant 
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potential, the originator drafts and submits a Project 
Nomination Proposal (PNP) to the IPO. The IPO then performs 
the "quick look" evaluation that is the subject of this 
research; those proposals evaluated likely to succeed are 
forwarded for notification to the Deputy Assistant Secretary 
of the Navy for International Policy (DASN(IP)). The DASN(IP) 
either sends the PNP back to IPO with feedback or submits the 
PNP to the Assistant Secretary of the Navy for Research, 
Development and Acquisition (ASN(RD&A)) for approval to 
Request Authority to Negotiate (RAN). Once ASN(RD&A) approval 
1s obtained, the IPO submits the PNP to the USD(A) for 
authority to negotiate in accordance with DoD Directive 
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A. 


III. PRESENTATION OF DATA 


BACKGROUND TO THE DEVELOPMENT OF SUCCESS CRITERIA 
1. Original Evaluation Procedure 


The IP Office originally used the following 20-point 


checklist to evaluate Project Nomination Proposals: 


MOU REQUIREMENT. Why must a MOU be employed? Is use of 
a MOU based on legal or policy grounds? Would use of 
another international program mechanism (e.g., FMS LOA, 
export license, etc.) achieve the same objectives? 


LEGAL. If a MOU is required, which statute(s) provide 
authority for the MOU? More specifically, what is the 
statutory authority for each of the obligations undertaken 
by the U. S. Government in the MOU? 


NUMBER OF PARTICIPANTS. How many nations desire to 
participate in the MOU? (Note that in general, bilateral 
agreements are encouraged.) 


MOU SCHEDULE. Can authority to negotiate be obtained and 
can a MOU be negotiated and concluded in time to meet 
desired program schedule(s)? 


PROGRAM OBJECTIVE. Is the objective of the MOU clearly 
stated? What is the motivation and benefit to the U. S. 
Government? What is the perceived motivation and benefit 
to the foreign government(s)? (Note that the combination 
of the two must create a "win-win" situation in order for 
the program to succeed.) 


PROGRAM REQUIREMENT. Is there a documented U. 5S. 
requirement which has: 


OPNAV support and/or approval? 
CINC/JCS support and/or approval? 
SYSCOM/Lab support and/or approval? 
RD&A support and/or approval? 

OSD support and/or approval? 
Congressional support and/or approval? 
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Is the combined U. S.-foreign requirement clearly stated 
in the MOU, and does it realistically reflect the needs of 
all parties? 


PROGRAM SCOPE OF WORK. Are proposed program tasks and 
associated schedules clearly stated? Will they achieve 
program objectives and meet program requirements in the 
areas of technical performance, cost and schedule? 


MANAGEMENT. Will this be a U. S. managed project, a 
foreign country managed project, or a jointly managed 
project? What is the management hierarchy (steering 
groups, program offices, etc.)? Will the program 
office(s) be integrated or not? 


FINANCIAL. What is the total cost of the project 
(financial and non-financial)? What is the U. S. share 
(financial/non-financial) versus the foreign share? What 
is the United States/foreign cost share rationale 
(production off-take, equal share, etc.), and are the 
contributions versus benefits equitable? What are the 
U. S. sources of funds/non-financial contributions? If 
DoN funds, are they in the budget/POM? If Nunn funds, 
have they been approved by DoN/OSD? If NRC waiver(s) are 
proposed, have they been approved by DoN/OSD and/or 
foreign governments involved? 


CURRENCY. Will the program be managed in U. S. dollars, 
foreign currency, or a mix? Will the establishment of a 
joint program bank account be required? 


CONTRACTUAL ARRANGEMENTS. Will U. 8S. contracedgal 
procedures be used or not? Who will serve as the U. S. 
contracting agency (if applicable)? What type of contract 
(FFP, CPIF, etc.) will be employed? Are there any special 
arrangements or procedures? If so, are they in accordance 
with the FAR? 


WORK-SHARE. If a cooperative MOU, what is the anticipated 
division of work-share among participants, by volume and 
by type (high tech, low tech)? Does the proposed work- 
Share align with cost-share (i. e. is it equitable)? 


OFFSET AGREEMENTS. Are there industrial offset agreements 
addressed in the MOU that are associated with this 
program? Will the U. S. Government play a role? Is that 
role consistent with DoD policy on offsets? 


INDUSTRIAL BASE IMPACT. What will be the impact on the 


U. S. industrial base (including sub-tier suppliers) of 
proposed work-share/offset agreements? 
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@® INTELLECTUAL PROPERTY RIGHTS. What background and 
foreground data rights are anticipated to be transferred 
EO LOre1gmeparticipants? Do U. S. rights align with U. S. 
EGE rIbutaons? (Note that for substantial U. S. 
contributions, we Should obtain a greater degree of data 
rights than our prospective partners.) 


@® TECHNOLOGY TRANSFER. Are sensitive technologies to be 
shared or transferred? Have such transfers been cleared 
through DON and OSD (DTSA)? Is TTSARB approval required? 
Has disclosure guidance to the negotiating team been 
developed and issued? Has a Delegation of Disclosure 
Letter (DDL) to the Program Manager (PM) been issued? 


® DISCLOSURE OF CLASSIFIED INFORMATION. Widen Uren S. 
classified information or material be disclosed to foreign 
nationals during the course of the program? Is approval 
by the National Disclosure Policy Committee (NDPC) 
required? (Note that both technology transfer and 
disclosure of CMI should be addressed in the MOU 
negotiating guidance and DDL.) 


® THIRD PARTY SALES AND TRANSFERS. Wiis sEvcuresW. —Se 
disclosures require concurrence of other participants? 
(Note that if the U. S. possesses considerable "background 
equity" in the program, permitting foreign "veto" of 
potential U. S. sales should be avoided.) 

® MOU CLARITY. Does the MOU clearly state the arrangements 
listed above or are they to be documented by side 
eqgreements? 

@® REQUIRED DOCUMENTATION. Have the following supporting 
documents been prepared: Legal Memorandum; Fiscal 
Memorandum, Technology Security Risk Assessment (or its 


successor, Technology Assessment and Control Plan); 
Industrial Base Factors Analysis? 


The IP Office halted use of the checklist since it offered an 


entirely subjective overview and since it could not provide a 


methodology for comparing one PNP to another. 
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2. OSD Decision Matrix 

The IP Office considered adopting the following four-part 
decision matrix used by the Office of the Secretary of Defense 
for selecting programs to pursue among those nominated for 


international research and development projects: 


SUITABILITY FOR HIGH (9 PTS) | MEDIUM (4 LOW (1 PT) 
COLLABORATION RED PTS) YELLOW GREEN 
STATUS OF MOU SIGNED ENVEROCESS PROCESS @HO® 
YET STARTED 


, EKER PHOODFOr CONSISTENT REQUIRES REQUIRES 
bocetan SUPPORT OR WITH FOREIGN | SOME CHANGE RADICAL DE- 
COMMITMENT APPROACHES IN APPROACH PARTURE IN 

TO DEFENSE APPROACH OR 
NEW POLICY 


3... SEMILART TY. Waiter mNCOREO- INCORPO: REQUIRES 
ONGOING PROJECT RATES MAJOR RATES SOME TERMINATION 
ONGOING ELEMENTS OF OR REDUCTION 
PROJECT ONGOING OF ONGOING 
RDT&E PROJECT 


4. LIKELIHOOD OF U. S. AND SIMILAR VERY 

SUCCESS IN ALLIES ARE SCHEDULE DIFFERENT 

COLLABORATION ON SAME SCHEDULES 
SCHEDULE 

A. TIMING 


Be COs > COULD MODERATE Hittite 
RADICALLY INFLUENCE IMPACT 
REDUCE COsS® 


Cy TECHNOBOG® POTENTIAL POTENTIAL LITTLE 

SHARING BOR SIGNIF - FOR MODERATE | POTENTIAL 
ICANT SAVINGS FOR SAVINGS 
SAVINGS 


De SECURITY UNCLASSIFIED | CONFIDENTIAL | SECRET; VERY 
NO SENSITIVE | MODERATELY SENSITIVE 
TECHNOLOGY SENSITIVE TECHNOLOGY 
OR SYSTEMS TECHNOLOGY OR SYSTEMS 
OR SYSTEMS 
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SUITABILITY FOR HIGH (9 PTS) MEDIUM (4 LOW (1 PT) 
COLLABORATION RED PTS) YELLOW GREEN 


E. RATIONALIZATION, | SUPPORTS 2UPPOR TS SUPPORTS 
STANDARDIZATION, AREAS WHERE AREAS WHERE AREA WHERE 
INTEROPERABILITY RSti1.S RSI IS MOD- Rot AS .Nor 
(es 1) CRUCIAL BRATELY VERY 
IMPORTANT IMPORTANT 


Sea DeErENSE INDUS. PROVIDES MODERATE LITTLE OR NO 

TRIAL BASE IMPACT SIGNIFICANT ACCESS ACCESS 
ACCESS TO 

A. ACCESS TO UNIQUE | KEY FOREIGN 

FOREIGN ASSETS ASSETS 


B. ACCESS TO UNIQUE | PROVIDES MODERATE SIGNIFICANT 
wes. ASSETS LITTLE OR NO | ACCESS ACCESS 
ACCESS TO 
KEY 
Us. ASskrs 


eet MiPACT ON, U 5. SUPPORTS LITTLE UNDERCUTS 

PRODUCTION LEVELS ee CIR NT IMPACT \OU.G oe 

(<5 YEARS) ak Goes PRODUCTION 
PRODUCTION LEVELS 
LEVELS 


PreelMPACT ON )U. S$. SCANS IIS BEAM bieyS UNDERCUTS 
PRODUCTION LEVELS EFPPICIENT IMPACT Ur 2. 

(>5 YEARS) Us S.. PRODUCTION 
PRODUCTION LEVELS 
LEVELS 


Peeve aAcCl ON U. S. ENHANCES LITTLE DEGRADES 
TECHNOLOGICAL LEAD Dos kA IMPACT U. S. LEAD 
IN MANTECH IN MANTECH 





POTENTIAL TO RE- 
DRESS CONVENTIONAL 
FORCE DEFICIENCY 









LOW (1 PT) 
GREEN 





















Fels” ale ™ Da So) 
RED 


MEDIUM (4 
PTS) YELLOW 





































1. NEW CONVENTIONAL | GIVES CAPA- REDRESSES PROVIDES 
DEFENSE CAPABILITY | BILITY TO LONG TERM INCREMENTAL 
REDRESS IMBALANCE IMPROVEMENT 

SHORT TERM IN NATO/ 
ALLIED CAPA- 


IMBALANCE 








——— BILITY 
SIGNED JCS 
AND SERVICE 
REQUIREMENT 


Zoe MEETS U.S. 
OPERATIONAL 
REQUIREMENT 


REQUIREMENT 
IN REVIEW 


NO FORMAL 
REQUIREMENT 
EXISTS 
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POTENTIAL TO RE- 


DRESS CONVENTIONAL 
FORCE DEFICIENCY 


ye 
NATO/ALLIED DEFI- 


ples 
RED 


(9 PTS) 


ADDRESSES FORMAL | FORMALLY 


CIENGZ 


PRIORITY WITHIN 
SERVICE/AGENCY 


EEE LEHOOD OF 


route ON 
ACQUISITION 


o- 


PLANNED COMMIT - 


APPROVED 
REQUIREMENT 


HIGH (9 PTS) 
RED 


MEETS VERY 
HIGH PRIGCRI- 
DY (SERVICE, 
AGENCY 
OBJECTIVE 


SOLIDLY IN 


MENT TO PROCUREMENT } LONG TERM 


3% 


WILLINGNESS TO 


BUND SPR 00" 
COs Ts 


PLANS 


SERVICE/ 
AGENCY WILL 
COVER ALL 
SUGH, COSas 


MEDIUM (4 
PTS) YELLOW 


LINKED TO 
| DEFICIENCIES 


IN NATO CMF 
OR OTHER 
FORMAL 
DOCUMENT 


MEDIUM 
PTS) YELLOW 


MEETS MODER- 
TEMA Sh MSD 5c EO va Elle 
OBJECTIVE 


MODERATE 
LEVEL OF 
LONG TERM 
COMMITMENT 


SERVICE/ 
AGENCY WILL 
COVER ONLY 
PART OF 

COS LS 


LOW (1 PT) 
GREEN 





| WEAK LINKAGE | 


TO FORMAL 
DOCUMENT 


LOW (1 PT) 
GREEN 


GENERALLY 

SUPPORT 
OF OBJEC] 

TIVES 


NOT oY Ew 
INCORPORATED 
IN LONG TERM 
PLANS 


SERVICE/ 
AGENCY WILL 
NOT COVER 
SUCH COSTS 


Seer S = OR UL Se WELL 
ek | ESTABLISHED 


WILLINGNESS TO 


| USE U. S. 
| FACILITIES 


| SUPPORT IN FIVE- 


YEAR DEFENSE PLAN 
a 


Ser On nh aN 


SERVICE POM 


ee 


oO POR any 


FOREIGN BUDGET 


| WILLING ao 


|} COMMIT KEY 


FACTELTIIES 


HIGH 
RED 


(PTS) 


CONTAINED IN 
SERVICE POM 


CONTAINED IN 


FOREIGN 
BUDGET 
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| May COMMIT 
KEY 


| FACILITIES 


MEDIUM 
PTS) YELLOW 


COMMITMENT 
INO, 1510/1 6 8 


POM LATER 


COMMITMENT 
Teun Th 
FOREIGN 
BUDGET 


| NOT WILLING 
TO COMP 
KEY FACILI - 
TLES 


LOW (1 PT) 
GREEN 


COMMITMENT 


vQoml: 








Though the OSD evaluation format did allow for a 
Quantitative comparison of one candidate program to another, 
the IP Office decided not to adopt the OSD format because it 
did not adequately consider the point of view of the foreign 
participants in the MOU. Their experience with MOUs convinced 
the IP Office that the key to the success of an MOU was the 
"degree of convergence" between the U. S. decision factors and 
the foreign decision factors; a high degree of convergence 
indicated a synergism at work in the MOU that helped ensure a 


successful program. 


B. INITIAL SUCCESS CRITERIA EVALUATION 

1. U. S./Foreign Evaluation Criteria 

On October 5, 1990, the IP Office developed the following 
weighted criteria by which a candidate program could be 


evaluated: 


LEVEL OF 
COMMITMENT : 


- CONGRESS / LEGISLATION MEMBER/STAFF | NO INTEREST 
EQUIVALENT 
aie y Peeiner or rents | 


-OSD/MOD DUSD/ASD UP WORKING 


LEVEL 


- SERVICE/ ASN(RDA) UP SPONSOR WORKING 
EQUIVALENT SYSCOM/PEO LEVEL 
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i IIT. PROGRAMMATIC 


FACTORS 


- PROGRAM MAJOR SYSTEM | OTHER SYSTEM | TECHBASE 
OBJECTIVE FLEET AND/OR 
DELIVERY DELIVERY STUDIES 


= PAR APPROVED OR APPROVED TOR | INTENT/ NO 
REQUIREMENTS OR REQUIRED 


- ALLIANCE APPROVED APPROVED INTENT TO 
REQUIREMENTS NATO OUTLINE | NATO MISSION | DEVELOP 
STAFF TARGET | NEED 

EVALUATION 


-MOU/ PROGRAM NO MOU/ CONSTRAINED MAJOR 

SCHEDULES PROGRAM MOU / PROGRAM SCHEDULE 
SCHEDULE SCHEDULES DISCONNECT 
PROBLEMS 


-INDUSTRIAL BASE ENHANCES NO IMPACT ON | NEGATIVE 

IMPACT USE sy? Ui eS 2 U2 Se 
INDUSTRIAL INDUSTRIAL INDUSTRIAL 
BASE BASE BASE IMPACT 


32RD PAR SAS DESIRE BROAD | DESIRE DESIRE NO 
OBJECTIVES SALES SELECTED SALES 
SALES 


Lit... Api LEY, AO ARQ 5 1 
CONTRIBUTE 
- FUNDING FULLY FUNDED | PARTIALLY INTENT «TO 
IN POM FUNDED IN POM 
POM 


- GOVERNMENT FULL SYSCOM/ | PARTIAL SYSCOM/LAB 
TECHNOLOGY LAB SUPPORT SYSCOM/LAB RESOURCE 
SUEPORS PROBLEMS 


PARTIAL 
SUPPORT/ 

TECHNOLOGY 
AVAILABLE 



























































Ni Sa 
TECHNOLOGY 


FULL INDUS - 
TRY SUPPORT/ 
TECHNOLOGY 

AVAILABLE 


INDUSTRIAL 
SUPPORT/ 

TECHNOLOGY 
PROBLEMS 


- TECHNOLOGY NO TECH | SELECTED MAJOR TECH 


TRANSFER TRANSFER TECH TRANS - TRANSFER 
ISSUES FER ISSUES ISSUES 


re SUSIE SS LOW RISK MEDIUM RISK HIGH Risk 
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2. Five-step Evaluation Method 


Using the above criteria, the IP office conducted the 


following five-step evaluation process on each Project 


Nomination Proposal: 


Betaplrshy initdal U. S. "Strength Rating" based on 
evaluation of proposed program vis-a-vis IP "program 
success" indicators. 


Analyze Foreign Intentions/Capabilities in a similar 
manner. 


Compare results of the first two steps to "match up" U. 
S./Foreign strengths/weaknesses. Since U. S. and foreign 
factors can either support, neutralize, or conflict with 
each other, the initial U. S. "Strength Rating" may be 
positively or negatively affected by the degree of 
convergence between the U. S. and Foreign intentions and 
capabilities. 


Qualitatively determine the proposed program’s 
"Probability of Success" and "Payoff Potential" if program 
is successful. 


Develop a consolidated priority listing which takes into 
account the program’s quantitative "Composite Strength 
Rating," and qualitative "Probability of Success" and 
"Payoff Potential" assessments. 


The individual criteria for each area (Level of 


Commitment, Programmatic Factors, Ability to Contribute) were 


assigned values from one to ten; an average for each area was 


then determined and multiplied by a predetermined weight 


mOn 75, 1.5, and 1.0 for Level of Commitment, Programmatic 


Factors, and Ability to Contribute, respectively) to arrive at 
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an area "Strength Rating." Area strength ratings for both the 
United States and the foreign participants were compared, 
summed and multiplied by a second weight to arrive at a 
composite area strength rating. This second weight (ranging 
in multiples of 0.25 from 0.50 to 1.50) was subjectively 
determined by the IP Office by aligning the individual 
criteria under each U. S. area with their foreign 
counterparts. If a positive degree of convergence existed for 
an area, the weight would be a value greater than one. A 
value of one was assigned for a neutral degree of convergence, 
and a value less than one was assigned for a negative degree 
of convergence. These composite area strength ratings were 
then summed to determine the final composite value by which 
programs would be ranked. 

3. Initial Application Of Evaluation Criteria and Method 

a. Programs Evaluated 
The following programs were initially evaluated; the short 


titles in parentheses are for cross-referencing purposes: 
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| PROGRAM TITLE FOREIGN PARTICIPANTS 


NATO MECHANICAL FRANCE, NETHERLANDS, 
MINESWEEPING SYSTEM SPAT, LTALY 
(NMMS ) 


ANTI-SHIP MISSILE UNITED KINGDOM 
COUNTERMEASURES 
(ASMCM) 


"SPINNAKER" (SPIN) CANADA 


te >. PACOM COMBINED AUSTRALIA, SOUTH KOREA, 
INTEROPERABILITY PROGRAM JAPAN 
(er?) 


NATO CSNI (CSNI) UNITED KINGDOM, FRANCE, 
NETHERLANDS, CANADA 


HMEONTINUOUS PROCESSING OF [if FRANCE 
SOLID PROPELLANTS 
(C2P2) 





b. Evaluation Sheets 
The individual evaluation sheets for the above programs 
are recreated on the following six pages. Although the format 
of these recreations differs slightly from the originals due 
to software limitations, the data and general sense of the 


evaluations remain true to the originals. 
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PROGRAM SHORT TITLE: NMMS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature al 
SiC Sy. 3 -MOD 6 
-Service 8 -Service/Equiv. 9 
-Strength Rating 300 -Strength Rating 4.00 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 4 -Program Objective 4 
-Military Reqm’ts 10 -Military Reqm’ts 10 
-Alliance Reqm’ts 10 -Alliance Reqm’ts 10 
-MOU/Program 5 -MOU/Program 5 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
tnpact iinipaece 
-3rd Party Sales Ss -3rd Party Sales 5 
Objectives Objectives 
-Strength Rating G75 -Strength Rating S275 
Degree of Convergence Weight: 1.25 
Aji 1 tyson Gontirs bute Ability to Contributes 
-Funding 10 -Funding 10 
-Government ae0) -Government 106, 
Technology Technology 
-Industry 8 Sb) Usneen 8 
Technology Technology 
-Technology ini@) -Technology 10 
Transfer Transfer 
-Security NA -Security 5 
-Strength Rating G2 50 -Strength Rating 8 266 
Degree of Convergence Weight: 1.5 
U.*S. Strengen 2225 Foreign Strength 22.35 


Probability of Success: Medium-High 


Payoff Potential: Medium-High 


COMPOSITE STRENGTH Sie pe re 
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PROGRAM SHORT TITLE: ASMCM 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature 1 
-OSD a -MOD 3 
-Service 6 -Service/Equiv. 5 
-Strength Rating 200 -Strength Rating 2.525 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 7 -Program Objective 7 
-Military Reqm’ts 10 -Military Reqm’ts S 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/ Program 10 -MOU/ Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
iinipact rmpact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating eee 0 -Strength Rating On 3 
Degree of Convergence Weight: 1.25 
Poawerty to Contribute (1.0) Ability tom@ont ri bute (1.0) 
-Funding 7 -Funding 7 
eoverniment ake =CovVerounent Ie 
Technology Technology 
-Industry NA -Industry NA 
Technology Technology 
-Technology 5 -Technology 5 
Transfer Transfer 
Ssecurity NA Security 10 
-Strength Rating 7) oe¥s -Strength Rating 8.00 
Degree of Convergence Weight: 1.5 
Wee, ot rength Zeleno S Foreign Strength PRG) Foes 


Probability of Success: High-High 
Payoff Potential: High-High 


COMPOSITE STRENGTH 54.91 
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PROGRAM SHORT TITLE: SPIN 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress a -Legislature a 
-OSD 3 -~MOD 5 
-Service 8 -Service/Equiv. 8 
-Strength Rating S06 -Strength Rating ei 5/0) 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 6 -Program Objective 6 
-Military Reqm’ts 10 -Military Reqm’ts 10 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program 9 -MOU/Program 9 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
impact inpace 
-3rd Party Sales NA -3rd) Party scales NA 
Objectives Objectives 
-Strength Rating 1 ees -Strength Rating pie 2/5 
Degree of Convergence Weight: 1.25 
Ability tor Cont rmouee. (ilu Ability to Contribute (ig 
-Funding 10 -Funding 10 
-Government 8 -Government 8 
Technology Technology 
-Industry 5 -Industry 5 
Technology Technology 
-Technology 7 -Technology 7 
Transfer Transfer 
-Security NA -Security ali) 
-Strength Rating yio8, -Strength Rating S00 
Degree of Convergence Weight: 1.25 
U. 9S. Strengen Zee 7S Foreign Strength 22005 


Probability of Success: Medium-High 
Payoff Potential: High-High 


COMPOSITE STRENGTH 54.00 
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PROGRAM SHORT TITLE: CIP 


Uwe STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress a -Legislature i 
-OSD 8 - MOD 8 
-Service ) -Service/Equiv. 9 
-Strength Rating 4.00 -Strength Rating 4.50 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 5 -Program Objective 5 
-Military Reqm’ts 8 -Military Reqm’ts 8 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program me -MOU/Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact iMpace 
ssa Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating iL). S10) -Strength Rating iO. 50 
Degree of Convergence Weight: 1.25 
pieeeey to Contribute (1.0) AiominseyeeOmeonisiluitce (1.0) 
-Funding 5 -Funding 9 
-Government 8 -Government 8 
Technology Technology 
-Imadustry NA -Industry NA 
Technology Technology 
-Technology 8 -Technology 8 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating Js One -Strength Rating Sen > 
Degree of Convergence Weight: 1.0 
Wes. Strength Py Nis i0) Foreign Strength 235.15 


Probability of Success: Medium-High 
Payoff Potential: Medium-High 


COMPOSITE STRENGTH 50.50 
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PROGRAM SHORT TITLE: CSNI 


UNITED STATES FOREIGN 
Level of Commitment (0.75 Level of Commitment (0.75) 
-Congress 3 -Legislature 2 
-OSD 4 - MOD 5 
-Service 6 -Service/Equiv. 6 
-Strength Rating B25 -Strength Rating 3.25 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5 Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 3 
-Military Regm’ts Z -Military Reqm’ts 3 
-Alliance Regm’ts 10 -Alliance Reqm’ts 10 
-MOU/Program 8 -MOU/Program 8 
Schedules Schedules 
-Industrial Base 5 -Industrial Base = 
Impact imnpac & 
-3rd Party Sales NA jGigel Jeo? Seles NA 
Objectives Objectives 
-Strength Rating yee -Strength Rating Siw 0 
Degree of Convergence Weight: 1.25 
Aba] 1 tyebopContrnibute Gig) Ability to Contribute (ig 
- Fundang 5 -Funding 10 
-Government da) -Government 10 
Technology Technology 
-Industry NA -Industry NA 
Technology Technology 
-Technology 10 -Technology 10 
TEanster Transfer 
-Security NA -Seeurity 5 
-Strength Rating G23 -Strength Rating S27 
Degree of Convergence Weight: 1.25 
U. S. Strengen toes 8 Foreign Strength 2 Oe 


Probability of Success: Medium-High 
Payoff Potential: Medium-Medium 


COMPOSITE STRENGTH 48.48 
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PROGRAM SHORT TITLE: C2P2 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature 1 
-OSD 2 -MOD 4 
-Service 4 -Service/Equiv. 6 
-Strength Rating 1.758 -Strength Rating Ze S 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts 1 -Military Regqm’ts 1 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program 6 -MOU/Program 6 
Schedules Schedules 
-Industrial Base Ie) -Industrial Base 18 
impact Impact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 65 -Strength Rating 6.75 
Degree of Convergence Weight: 1.0 
Piemmecy tO Contribute (1.0) BilimGiiVveeOumeemiaiioulce (1.0) 
-Funding 2 -Funding 8 
-Government I -Government a6, 
Technology Technology 
slmaustry 7 =) GS iso y- ¢ 
Technology Technology 
-Technology 8 Pe echmie log y 8 
Transfer Transfer 
-Security NA -Security 5 
-Strength Rating Des -Strength Rating 7260 
Degree of Convergence Weight: 1.25 
eo. otrengeh 25 Foreign Strength pa gees 6 


Probability of Success: Medium-Medium 
Payoff Potential: Medium-High 


COMPOSITE STRENGTH SDs 


as 


c. Results of Initial Application 
The following table summarizes the results of this initial 


evaluation: 


COMPOSI THRU. Ss. FOREIGN PROBABIL- | PAYOFF 
STRENGTH STRENGTH | STRENGTH | ITY OF POTEHRE 
RATING RATING SUCCESS TIAL 


5 Sige 3 2 2:5 22435 MEDIUM - HIGH 
——— 


-ASMCM 54. or yA ag 21.330 20.38 Shi e. fo HIGH 


SPIN 54.00 21.75 22.75 MEDIUM- HIGH 
HIGH 
CIP 54.00 ae Exo 23.75 MEDIUM- MEDIUM- 
e HIGH HIGH 
| CSNI 48.48 Toe ae 20.70 MEDIUM- MEDIUM 
HIGH 


C222 So 4 1a Ae. aly apa) MEDIUM MEDIUM- 
HIGH 





4. Second Application of Evaluation Criteria and Method 
a. Programs Evaluated 
An additional 12 programs were evaluated later in the same 


fiscal year. These programs are listed below: 
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FOREIGN PARTICIPANTS 


PROGRAM TITLE 


REMOTE SENSING UNIT WITH NORWAY, 
SYNTHETIC APERTURE RADAR UNITED KINGDOM 
(RSU) 


AIRBORNE ELECTROMAGNETIC CANADA 
SENSOR (AES) 


TACTICAL LIGHTWEIGHT 
SURVEILLANCE SATELLITE 
(PiGSs ) 


TACTICAL TARGET SUPPORT AUSTRALIA 
er TS) 


DISTRIBUTED TACTICAL UNITED KINGDOM 
SURVEILLANCE (DTS) 


































AUSTRALIA 





TACTICAL SHIP UNITED KINGDOM 
SURVEILLANCE (TSS) 


OCEANOGRAPHY COOPERATION | JAPAN 
fee ) 
OCEANOGRAPHY COOPERATION | NORWAY 
(OC-N) 


OCEAN COLOR ASSESSMENT, UNITED KINGDOM, NORWAY, 
Sa TSN SEA TOCA=6) ICELAND, DENMARK 






OCEAN COLOR ASSESSMENT, UNITED KINGDOM, NORWAY, 
MEDITERRANEAN SEA ICELAND, DENMARK 


(OCA-M) 


MILLIMETER WAVE SEA | GERMANY, UNITED KINGDOM 
| SURVEILLANCE (MWSS) 


REMOTE SPECTROMETRY CANADA, UNITED KINGDOM 
RESEARCH (RSR) 


b. Evaluation Sheets 





The following pages replicate the evaluation sheets 
resulting from this second application of the success criteria 


and 5-step evaluation method to candidate programs: 
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PROGRAM SHORT TITLE: RSU 


UNEERE DS SSTATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress a -Legislature aL 
-O5) aL - MOD IL 
-Service 8 -Service/Equiv. 8 
-Strength Rating a 8 -Strength Rating 2950 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 5 -Program Objective 5 
-Military Reqm’ts 10 -Military Regm’ts 10 
-Alliance Regqm’ts 5 -Alliance Regqm’ts 5 
-MOU/Program 7 -MOU/ Program 7 
schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact Impact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 2) 1510 -Strength Rating or6 0 
Degree of Convergence Weight: 1.25 
Abid i tymibemiCont mastic meen) Ability to Contributes 
-Funding y -Funding 5 
-Government 9 -Government 9 
Technology Technology 
- Imduseny 10 -Industry LG 
Technology Technology 
-Technology Me -Technology 10 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating 9200 -Strength Rating 8.80 
Degree of Convergence Weight: 1.25 
Us? Sse Strength Zae8 Foreign Strength 20539 


Probability of Success: Medium-High 
Payoff Potential: Medium-High 


COMPOSITE STRENGTH SI 4s, 
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PROGRAM SHORT TITLE: AES 


UNTTEDYSTALES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75 
-Congress 1 -Legislature 1 
OLS 1B, al - MOD a 
-Service 4 -Service/Equiv. 5 
-Strength Rating i 50 -Strength Rating eds 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 3 -Program Objective 3 
-Military Reqm’ts 3 -Military Regm’ts 3 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program 10 -MOU/Program 10 
Schedules Schedules 
-Industrial Base 4 -Industrial Base 4 
impact impace 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 50 -Strength Rating U5 0 
Degree of Convergence Weight: 1.25 
pub y to Contribute (1.0) Abrilityeveméentribute (1.0) 
-Funding 7 -Funding a 
-Government 10 -Government 10 
Technology Technology 
srmaustry 10 -Ineus tisy ilg6, 
Technology Technology 
-Technology AG, -Technology 16 
Transfer Transfer 
“Security NA -Security ae 
-Strength Rating CNRS -Strength Rating 9.40 
Degree of Convergence Weight: 1.25 
ees. Strength e225 Foreign Strength 1365 


Probability of Success: Medium-High 
Payoff Potential: Medium 


COMPOSITE STRENGTH 45.31 
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PROGRAM SHORT TITLE: TLSS 


UNDEEEDSSTATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature a 
-OSD al - MOD i 
-Service 5 -Service/Equiv. 5 
-Strength Rating Tes -Strength Rating 1 hd 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 7 -Program Objective 7 
-Military Reqm’ts 1 -Military Regm’ts al 
-Alliance Reqm’ts i -Alliance Reqm’ts 7 
-MOU/Program 10 -MOU/Program In 
Schedules Schedules 
-Industrial Base 7 -Industrial Base 4 
impact Impact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating S60 -Strength Rating 8.70 
Degree of Convergence Weight: 1.25 
Abaisdlityaieo. Commmupuisee( 1. 0) Ability to Contribute. (iis 
-Funding 2 -Funding 2 
-Government 2G) -Government 10 
Technology Technology 
-THaustry IG SmMGgustry 4 
Technology Technology 
-Technology 5 -Technology 5 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating 65> -Strength Rating 6220 
Degree of Convergence Weight: 1.25 
U.S Strengiteh LBrelso Foreign Strength 16 465 


Probability of Success: Medium-High 
Payoff Potential: Medium-High 


COMPOSITE STRENGTH 42.56 


Si 


PROGRAM SHORT TITLE: TTS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature BY 
-OSD 1 -MOD 1 
-Service 5 -Service/Equiv. 5 
-Strength Rating aa sles -Strength Rating je os 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5 Programmatic Factors (1.5) 
-Program Objective 5 -Program Objective 5 
-Military Reqm’ts 2 -Military Reqm’ts 2 
-Alliance Reqm’ts 7 -Alliance Reqm’ts 7 
-MOU/ Program 8 -MOU/Program 8 
Schedules Schedules 
-Industrial Base 6 -Industrial Base 6 
impact Impact 
soma Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 8.40 -Strength Rating 8.40 
Degree of Convergence Weight: 1.25 
Piety to Contribute (1.0) Ao lityveco Cemtribute (1. 0} 
-Funding 1 -Funding 1 
-Government age -Government LO 
Technology Technology 
-Industry ANS; -fuGusery 10 
Technology Technology 
-Technology > -Technology 5 
meansfer Transfer 
-Security NA -Security ang) 
-Strength Rating 6.50 -Strength Rating TRAPS 
Degree of Convergence Weight: 1.25 
Mees. Strength 16.65 Foreign Strength Bg Rae 


Probability of Success: Medium-High 
Payoff Potential: Medium 


COMPOSITE STRENGTH 615563 


Spe) 


PROGRAM SHORT TITLE: DTS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 2 -Legislature at 
-OSE ae - MOD ae 
-Service 5 -Service/Equiv. 5 
-Strength Rating Ly! -Strength Rating 1.5 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 2 -Program Objective 2 
-Military Reqm’ts 10 -Military Reqm’ts 10 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program , -MOU/Program 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact Impact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 9.00 -Strength Rating 9. 00 
Degree of Convergence Weight: 1.00 
Ability to Contribute (1.0) Ability to Contribute (gp 
-Funding 5 -Funding S 
-Government 8 -Government 8 
Technology Technology 
-Industry 5 -Industry 5 
Technology Technology 
- Technology 16, -Technology 10 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating Tae -Strength Rating 7.60 
Degree of Convergence Weight: 1.00 
Us-Ss Strengieh ee Foreign Strength 18335 


Probability of Success: Medium 
Payoff Potential: Medium 


COMPOSITE STRENGTH SiGe) 
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PROGRAM SHORT TITLE: TSS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature a 
-OSD 1 -MOD Hi 
-Service 5 -Service/Equiv. 5 
-Strength Rating gas -Strength Rating Te 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts 10 -Military Reqm’ts 10 
-Alliance Reaqm’ts NA -Alliance Reqm’ts NA 
-MOU/Program 8 -MOU/Program 8 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact IGuiesvole 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 9.00 -Strength Rating o700 
Degree of Convergence Weight: 1.00 
Piomient yetso Contribute (1.0) Aibatl 1 tysiomeont ri bute )(1 0) 
-Funding 6 -Funding 4 
-Government ZO -Government 10 
Technology Technology 
mlncustry 5 -Industry S 
Technology Technology 
meechnology 6 -Technology 10 
Transfer Transfer 
-Security NA -Security ake) 
-Strength Rating eS ES: -Strength Rating 7.80 
Degree of Convergence Weight: 0.75 
Seas. Strength 16250 Foreign Strength LS). 5 


Probability of Success: Medium 
Payoff Potential: Medium-Low 


COMPOSITE STRENGTH B54 16 
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PROGRAM SHORT TITLE: OC-J 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature il 
-OSD ab -MOD a 
-Service 5 -Service/Equiv. di. 
-Strength Rating lie? 5 -Strength Rating Ome 5 
Degree of Convergence Weight: 1.0 
PrOgranitadt tepmaceOucm clas) Programmatic Factors (1.5) 
-Program Objective 2 -Program Objective 2 
-Military Reqm’ts 3 -Military Reqm’ts 3 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/ Program 10 -MOU/ Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
iMac C Impace 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating TiO -Strength Rating pee) 
Degree of Convergence Weight: 1.00 
Abi ah VerouCeminaib lipemia) Ability to Cont ribute (ae 
-Funding al -Funding il 
-Government 10 -Government 10 
Technology Technology 
-Industry NA ~lnGgustery NA 
Technology Technology 
-Technology ce -Technology 10 
Transfer Transfer 
-Security NA -Security ae 
-Strength Rating HOO -Strength Rating hase) 5 
Degree of Convergence Weight: 1.00 
U. S. Strength Gye. Foreign Strength LEE OO 


Probability of Success: Medium 
Payoff Potential: Medium 


COMPOSITE STRENGTH 3 Zea 
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PROGRAM SHORT TITLE: OC-N 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature 1 
-OSD al -MOD al 
-Service 5 -Service/Equiv. al 
-Strength Rating i725 -Strength Rating 02:75 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 2 -Program Objective 2 
-Military Reqm’ts Ss) -Military Reqm’ts 3 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/ Program 10 -MOU/ Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact imocae ce 
-3rd Party Sales NA "3b PaGeyeoales NA 
Objectives Objectives 
-Strength Rating Te 5e -Strength Rating 750) 
Degree of Convergence Weight: 1.00 
Domenie y to Contribute (1.0) AbaLey stoMeencribute (1 308 
-Funding sh -Funding al 
-Government 10 -Government ab) 
Technology Teennelogy 
ZENGuUSEry NA Sn Guisie ry, NA 
Technology Technology 
-Technology ak¢ -Technology TG, 
Transfer Transfer 
-Security NA -Security 156 
-Strength Rating 7200 -Strength Rating (ees 
Degree of Convergence Weight: 1.00 
ees. Strength 625 Foreign Strength 700 


Probability of Success: Medium 
Payoff Potential: Medium 


COMPOSITE STRENGTH BZ 2 5 
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PROGRAM SHORT TITLE: OCA-G 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature a. 
= (07:0) al -MOD 1 
-Service 4 -Service/Equiv. a 
-Strength Rating ie Se -Strength Rating 0. 5 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts Z -Military Reqm’ts 2 
-Alliance Regm’ts 2 -Alliance Reqm’ts Z 
-MOU/Program 10 -MOU/ Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
impact moe CE 
-3YGe Pa reyeooles NA eeses| Isenec, Selles NA 
Objectives Objectives 
-Strength Rating 60.0 -Strength Rating 6.00 
Degree of Convergence Weight: 1.00 
Abit yaboORG ontic bum aes) Ability to Contribute Giigy 
-Funding 6 -Funding & 
-Government 10 -Government 10 
Technology Technology 
-Industry NA - Indusiney NA 
Technology Technology 
- Technology 1G -Technology 10 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating 8.67 -Strength Rating Sicats 
Degree of Convergence Weight: 1.00 
U. S. Strength Gla Foreign Strength 15356 


Probability of Success: Medium 
Payoff Potential: Medium-Low 


COMPOSITE STRENGTH S267 
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PROGRAM SHORT TITLE: OCA-M 


UNITED STATES FOREIGN 

Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress i -Legislature uf 
-OSD al -MOD al 
-Service 4 -Service/Equiv. ae 
-Strength Rating a Si0 -Strength Rating OPS 


Degree of Convergence Weight: 1.0 


Programmatic Factors (1.5) Programmatic Factors (1.5) 


-Program Objective 1 -Program Objective 1 

-Military Reqm’ts 2 -Military Reqm’ts 2 

-Alliance Reqm’ts ie -Alliance Reqm’ts 2 

-MOU/Program 10 -MOU/Program 10 
Schedules Schedules 

-Industrial Base S -Industrial Base 5 
Impact Impact 

-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 

-Strength Rating S200 -Strength Rating 6.00 


Degree of Convergence Weight: 1.00 


Fimmijtiy to Contribute (1.0) Ability item@entribute (1.0) 
-Funding 6 -Funding 5 
-Government 10 -Government 100, 
Technology Technology 

sMaust ry NA -Industry NA 
Technology Technology 

-Technology 10 -Technology 10 
Transfer Transfer 

-Security NA -Security 10 

-Strength Rating 8.67 -Strength Rating Seis 


Degree of Convergence Weight: 1.00 
wees. Strength aS 3 aL Foreign Strength ais aes) €) 
Probability of Success: Medium 
Payoff Potential: Medium-Low 


COMPOSITE STRENGTH Say, 


oe 


PROGRAM SHORT TITLE: MWSS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (O78 
-Congress a -Legislature 1 
-OSD al -MOD al. 
-Service 5 -Service/Equiv. 5 
-Strength Rating le 5 -Strength Rating es 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts 4 -Military Reaqm’ts 4 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-MOU/ Program 10 -MOU/Program 10 
Schedules Schedules 
-Industrial Base 8 -Industrial Base 5 
Impact Impact 
-3rd Party Sales NA -3rd Party Sales NA 
Objectives Objectives 
-Strength Rating 8263 -Strength Rating fray 8. 
Degree of Convergence Weight: 1.00 
Aba. satamtonG onimanion tem aeEO) Ability to Contribute Gigs 
-Funding 1 -Funding al 
-Government 10 -Government 10 
Technology Technology 
“TGS tas 8 -THAUSELY 8 
Technology Technology 
-Technology 3 -Technology 5 
Transfer Transfer 
-Security NA -Security 5 
-Strength Rating 6 ae0 -Strength Rating 5: 80 
Degree of Convergence Weight: 1.00 
U. S. Strength 3.8 Foreign Strength 15305 


Probability of Success: Medium 
Payoff Potential: Medium-Low 


COMPOSITE STRENGTH Bi ye 
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PROGRAM SHORT TITLE: RSR 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature a 
-OSD a -MOD 1 
-Service 5 -Service/Equiv. 5 
-Strength Rating Ls -Strength Rating are As: 
Degree of Convergence Weight: 1.0 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts al -Military Regqm’ts 1 
-Alliance Reqm’ts NA -Alliance Reqm’ts NA 
-~MOU/Program 10 -MOU/ Program 10 
Schedules Schedules 
-Industrial Base 5 -Industrial Base 5 
Impact Impact 
-3rd Party Sales NA Je, Pateye sales NA 
Objectives Objectives 
-Strength Rating 6236 -Strength Rating 6236 
Degree of Convergence Weight: 1.00 
pode y CO Contribute (1.0) Poaicy ho Comers pubes 0) 
-Funding a -Funding 2 
-Government 8 -Government 8 
Technology Technology 
-iIndustry 5 -Industry S. 
Technology Technology 
-Technology > —AUsolayarelioleny 5 
Transfer Transfer 
-Security NA -Security 10 
-Strength Rating Ped is -Strength Rating 6 @0 
Degree of Convergence Weight: 1.00 
Wes. olrength aL Asis Foreign Strength 14.13 


Probability of Success: Medium 
Payoff Potential: Medium 


COMPOSITE STRENGTH 2700 


Gr 


c. Results of Second Application 


This second set of program evaluations iS Summarized in 


the table below: 










COMPOS T BER Ue s- FOREIGN PROBABIL- | PAYOFF 
STRENGTH STRENGTH | STRENGTH | ITY OF POTEDS 
RATING RATING SeCeros TIAL 


RSU 5ae2S 2 4 0 20 0 MEDIUM- MEDIUM- 
HIGH HIGH 
AES Io. o dig gy Apa) ome) MEDIUM- MEDIUM 
HIGH 
iiss 42.56 Bree 0) 1 'Gr6 3 MEDIUM- MEDIUM- 
HIGH HIGH 
41.63 16.65 27235 MEDIUM- MEDIUM 
— 















iene 10 7. ‘Pes | ne 25 /MEDIUM | MEDIUM 
Sou: lo iis ano ilo Brahe) MEDIUM MEDIUM- 
— 
loca J Sac al Z5 Gr Tee) 16.00 00 /MEDIUM /MEDIUM 


32.25 16.00 | meprum _| meDIum 


OC2G 3ae6 7 dG eae] 18.55 MEDIUM MEDIUM- 
LOW 

oca-m — | 31.67 raf 25-50 | MEDzOM MEDIUM- 
LOW 


MWSS bt lean ieee 15.08 MEDIUM MEDIUM- 
LOW 


27.00 14.13. | MEDIUM | MEDIuM- 
LOW 
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C. REVISED SUCCESS CRITERIA EVALUATION 

1. Revised U. S./Foreign Evaluation Criteria 

On July 30, 1991, the IP office revised the evaluation 
criteria. The revision separated two criteria ("Technology 
Geanster" and "Security") from the "Ability to Contribute” 
area and established them in a new area, "Technology/Security 
Factors." Under the "Programmatic Factors" area, the revision 
deleted one criterion ("Alliance Requirements"), replaced two 
criteria ("Industrial Base Impact" and "3rd Party Sales 
Objectives") with new ones ("Workshare" and "Payoff"), and 
added NATO operational requirements criteria to the "Military 


Requirements" section. The revised worksheet is provided 


below: 










LEVEL OF ile 
COMMITMENT : 


- CONGRESS / LEGISLATION MEMBER/STAFF | NO INTEREST 
BQUIVALENT 
USD 


-OSD/MOD DUSD/ASD UP WORKING 
LEVEL 


- SERVICE/ ASN(RDA) UP SPONSOR WORKING 
EQUIVALENT SYSCOM/ PEO LEVEL 





Lays 
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II. PROGRAMMATIC 
FACTORS 


- PROGRAM 
OBJECTIVE FLEET FLEET 
DERIVERY DELIVERY 


-MILITARY APPROVED APPROVED INTENT/NO 
REQUIREMENTS OR/NSR TOR/NST OR REQUIRED 


-MOU/ PROGRAM NO MOU/ CONSTRAINED MAJOR 

SCHEDULES PROGRAM MOU / PROGRAM SCHEDULE 
SCHEDULE SCHEDULE DISCONNECT 
PROBLEMS 


-WORKSHARE WORKSHARE WORKSHARE WORKSHARE 
EQUALS W/IN PLUS DOES NOT 

COSTSHARE MENUS. 10% EQUAL 
COSTSHARE COSTSHARE 


AY OLE GREATLY ENHANCES REDUNDANT 
ENHANCES FLEET SYSTEM 
|S be) Sh Be PE Cay bas 
Eee Eee fie NESS 
NESS 


elTiTe “ARB IEITY TO 1 
CONTRIBUTE 


- FUNDING FULLY FUNDED | PARTIALLY INTENT TO 
IN POM FUNDED IN POM 
POM 


- GOVERNMENT FULL SYSCOM/ | PARTIAL SYSCOM/LAB 
TECHNOLOGY | LAB SUPPORT |! SYSCOM/LAB | RESOURCE 
SUPPORT PROBLEMS 

































TECHNOLOGY / 10 5 sl 
SECURITY 
FACTORS 








Dae eee, 


- TECHNOLOGY NO TECH SELECTED MAJOR TECH 
TRANSFER TRANSFER TECH lRANS= TRANSFER 
[SSUES FER ISSUES ISSUES 


-“SeCUR Ty LOW RISK MEDIUM RISK HIGH RISE 





70 


2. Revised Evaluation Method 

The evaluation process associated with this new set of 
criteria differs from the initial process. The weights (0.75, 
1.50, and 1.00) of the initial process are again assigned to 
the areas of "Level of Commitment," "Programmatic Factors," 
and "Ability to Contribute," and a weight of 1.00 is assigned 
to the new area "Technology/Security Factors." Ue Ss = betel 
foreign strength ratings are computed as before, but the 
composite strength rating computation differs. Under the 
initial process, each area’s strength ratings (U. S. and 
foreign) were first summed and then multiplied by subjectively 
assigned weights; the resultant products were then summed to 
arrive at the composite strength rating. Under the revised 
procedure, all area strength ratings (both U. S. and foreign) 
are summed and multiplied by a single, subjectively assigned 
weight factor. Summary U. S. and foreign strength ratings are 
not computed under the revised process; neither are 
qualitative measures given to "Probability of Success" or 
"Payoff Potential." 

3. Application of Revised Evaluation Criteria and Method 

a. Programs Evaluated 

Four programs were evaluated using the revised criteria 

and procedures. The following table identifies these 


weograms : 


vil 


PROGRAM TITLE FOREIGN PARTICIPANTS 


| MARITIME UNMANNED AERIAL CANADA, FRANCE, GERMANY, 
| VEHICLE (MUAV) NETHERLANDS, NORWAY, 
| UNITED KINGDOM 


| ADVANCED STEEL SYSTEMS 
(ADV ST) 


ALUMINUM OXYGEN BATTERIES | CANADA 
(AL BAT) 


REMOTE ATMOSPHERIC AND AUSTRALIA, BELGIUM, 
TONGCSPHERIC PEtEC Tier CANADA, GERMANY, SWEDEN 


SYSTEM (RAIDS) 





b. Evaluation Sheets 


The evaluation sheets representing these four programs are 


reproduced on the following pages: 


UD 


PROGRAM SHORT TITLE: MUAV 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature 0 
SC) S18) i - MOD 1 
-Service 6 -Service/Equiv. 6 
-Strength Rating 2200 -Strength Rating 26S 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 8 -Program Objective 8 
-Military Regqm’ts 8 -Military Reqm’ts 7 
-MOU/Program 6 -MOU/Program a0 
Schedules Schedules 
-Workshare 5 -Workshare 5 
-Payoff 8 -Payoff 8 
-Strength Rating ele 7 O -Strength Rating ie 0 
Por lity to Contribute (1.0) ion. 1 1 tygieoeGonmt r il biitseiiin 0) 
-Funding 10 -Funding 5 
-Government 0 -Government 0 
Technology Technology 
-Strength Rating 10.00 -Strength Rating 5-00 
Technolo Securit slag Technolo Securit er) 
-Technology IL.) -Technology 10 
Transfer Transfer 
-Security 10 -Security 0 
-Strength Rating ree 00 -Strength Rating O02 00 


OVERALL STRENGTH MATCHING 1.00 


COMPOSITE STRENGTH 62.73 


TS 


PROGRAM SHORT TITLE: ADV ST 


UNITED STATES 


Level of Commitment (0.75) 


-Congress AL 
JOSm ui 
-Service 3 
-Strength Rating 1 


Programmatic Factors (1.5) 


oe 


-Program Objective 6 
-Military Regqm’ts 3 
-MOU/Program 2 
Schedules 
-Workshare 0 
Hey Oar 8 
-Strength Rating “i 


Abi litveto: Contrabute (120) 


-Funding 5 

-Government 10 
Technology 

-Strength Rating y250 


Technolo Securit oe) 


-Technology ) 
Transfer 

-Security 10 

-Strength Rating S} A EI6 


FOREIGN 


Level of Commitment (0.75) 


-Legislature AL 
—Mee 5 
-Service/Equiv. 0 
-Strength Rating Z 


Programmatic Factors (1.5) 


-Program Objective 
-Military Regqm’ts 
-MOU/Program 
Schedules 
-Workshare 0 
shay Olt 4) 
-Strength Rating 2 


Ability to Contribute {aie 


25 


GeO WN 


.00 


-Funding 0 

-Government 8 
Technology 

-Strength Rating 3h 1ON0) 


Technolo Securit ae 


-Technology S, 
Transfer 

-Security 0 

-Strength Rating 9700 


OVERALL STRENGTH MATCHING 1.25 


COMPOSITE STRENGTH 58.00 


PROGRAM SHORT TITLE: AL BAT 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress 1 -Legislature a 
-OSD 3 -MOD Ht 
-Service 5 -Service/Equiv. a 
-Strength Rating 225 -Strength Rating O75 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 5 -Program Objective 0 
-Military Reqm’ts 10 -Military Reqm’ts 0 
-MOU/ Program 5 -MOU/Program 5 
Schedules Schedules 
-Workshare 0 -Workshare 2 
mecy OT go Pay ole z 
-Strength Rating 110) 5 feits: -Strength Rating aee5.0 
Aiea ty tosContribute. (1.0) Abbadon aieie@mCemit ri bieema eae) 
-Funding ane -Funding ii 
-Government 310 -Government 0 
Technology Technology 
-Strength Rating irom 00 -Strength Rating 7200 
Technolo Securit 1G Technolo Securit eo 
-Technology 10 -Technology 1 
Transfer Transfer 
-Security 0 -Security an 
-Strength Rating Oe.0 0 -Strength Rating O00 


OVERALL STRENGTH MATCHING 1.00 


COMPOSITE STRENGTH 55.38 


Us 


PROGRAM SHORT TITLE: RAIDS 


UNITED STATES FOREIGN 

Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature i 
=O5 0 al -MOD dL 
-Service 2 -Service/Equiv. at 
-Strength Rating 1.00 -Strength Rating 0; 75 


Programmatic Factors (1.5) Programmatic Factors (1.5) 


-Program Objective 2 -Program Objective 5 
-Military Regm’ts 8 -Military Regqm’ts 0 
-MOU/Program 8 -MOU/Program 0 
Schedules Schedules 
-Workshare 0 -Workshare 0 
-Payofft = -Payofft Z 
-Strength Rating Ses -Strength Rating 55 
Ability tosGemtrabure eine) Ability to Contribute ag 
- Funding 5 -Funding 3 
-Government 10 -Government 1a@ 
Technology Technology 
-Strength Rating TSO -Strength Rating Cae 
Technolo Securit fee) Technolo Securit der) 
-Technology 5 -Technology 5 
Transfer Transfer 
-~Security 5 SSeCur ary, 0 
-Strength Rating 500 -Strength Rating S20) 


OVERALL STRENGTH MATCHING 1.00 


COMPOSITE STRENGTH 39.25 
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c. Re-evaluation of Programs Using Revised Evaluation 
Criteria and Method 

Attempting to standardize the evaluation process, the IP 

office re-evaluated the six programs initially evaluated 

(NMMS, ASMCM, SPIN, CIP, CSNI, and C2P2) using the new 

criteria and procedures. These re-evaluations are reproduced 


on the following pages: 


a7 


PROGRAM SHORT TITLE: NMMS 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature i 
-OSD s - MOD 6 
-Service 8 -Service/Equiv. 9 
-Strength Rating 3.00 -Strength Rating 4.00 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 4 -Program Objective 4 
-Military Reqm’ts 10 -Military Reqm’ts 10 
-MOU/Program 5 -MOU/Program a0) 
Schedules Schedules 
-Workshare 5 -Workshare 5 
-Payofft 8 = lersl@On8 It 8 
-Strength Rating 9360 -Strength Rating nigel 10 
Ability stovontribmbe (1.0) Ability to Contribute sia 
-Funding 10 -Funding 10 
-Government 219, -Government 10 
Technology Technology 
-Strength Rating 10.00 -Strength Rating 1026 
Technolo Securit eo Technolo Securit 120 
-Technology 10 - Technology 10 
Transfer Transfer 
-Security 5 -Security 0 
-Strength Rating Taso -Strength Rating 10200 


OVERALL STRENGTH MATCHING 1.50 


COMPOSITE STRENGTH 75.20 
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PROGRAM SHORT TITLE: ASMCM 


UNITED STATES FOREIGN 
Level of Commitment (0.75) Level of Commitment (0.75) 
-Congress al -Legislature al 
-OSD al -MOD 3 
-Service 6 -Service/Equiv. 5 
-Strength Rating 2.00 -Strength Rating 2225 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 7 -Program Objective 7 
-Military Reqm’ts 10 -Military Reqm’ts 5 
-MOU/Program 10 -MOU/Program 10 
Schedules Schedules 
-Workshare 5 -Workshare 5 
aeeny Olt 10 = ely Cie 10 
-Strength Rating 276.0 -Strength Rating Hea ners be 8 
Ability to Contribute (1.0) Bbmigie to Contribute (420) 
-Funding 7 -Funding 7) 
-Government 120 -Government 10 
Technology Technology 
-Strength Rating 8.50 -Strength Rating 8250 
meehnolo Securit 1.0 Technolo Securit 16 
-Technology 5 -Technology 5 
Transfer Transfer 
-Security AG -Security 0 
-Strength Rating 500 -Strength Rating 5200 


OVERALL STRENGTH MATCHING 1.50 


COMPOSITE STRENGTH 65.95 


gS, 


PROGRAM SHORT TITLE: SPIN 


UNITED STATES FOREIGN 
Level or @onmi tment (0275) Level of Commitment (0.75) 
-Congress a -Legislature i 
-OSD 3 -MOD 2 
-Service 8 -Service/Equiv. 8 
-Strength Rating 3700 -Strength Rating 3350 
Programmatic Factors (1.5) Programmatic Factors (1.5) 
-Program Objective 6 -Program Objective 6 
-Military Regqm’ts 10 -Military Regm’ts 10 
-MOU/ Program 9 -MOU/Program 9 
Schedules Schedules 
-Workshare 5 -Workshare 5 
= 12th AG 16 18 16, -Payert 10 
-Strength Rating 2 00 -Strength Rating 1220e 
Abilaty stoesGontrmburem Gn) Ability to Contribute (aap 
- Funding 10 -Funding 10 
-Government 8 -Government 8 
Technology Technology 
-Strength Rating B00 -Strength Rating 9.00 
TeehnoLe Securit plier (0) Technolo Securit 18 
-Technology 7 -Technology a 
Transfer Transfer 
-Security 10 -Security 0 
-Strength Rating S450 -Strength Rating 1TaOe 


OVERALL STRENGTH MATCHING 1.25 


COMPOSITE STRENGTH 68.50 
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PROGRAM SHORT TITLE: CIP 
UNITED STATES 


Level of Commitment (0.75) 


seongress ab 
“(015.8 8 
-Service v] 
-Strength Rating 4 


Programmatic Factors (1.5) 


-Program Objective 5 
-Military Reqm’ts 8 


2OO 


-MOU/Program ine -MOU/ Program 10 
Schedules Schedules 
-Workshare 5 -Workshare 5 
eeayoOrt 8 Sy @ ie 8 
-Strength Rating 10.80 -Strength Rating Ors 0) 
Ability to Contribute (1.0) Bbility to Gomeribuce (1.0) 
“Funding 5 -Funding 9 
-Government 8 -Government 8 
Technology Technology 
-Strength Rating 6.50 -Strength Rating B50 
Technolo Secu ne Technolo Securit lig 
-Technology 8 -Technology 8 
Transfer Transfer 
-Security Ie -Security 0 
-Strength Rating S00 -Strength Rating 8.00 


FOREIGN 


Level of Commitment (0.75) 


-Legislature il 
-MOD 8 
-Service/Equiv. 9 

4 


-Strength Rating a9) 


Programmatic Factors (1.5) 


-Program Objective 5 
-Military Regqm’ts 8 


00 


OVERALL STRENGTH MATCHING 1. 


COMPOSITE STRENGTH 


sul 


62.10 


PROGRAM SHORT TITLE: CSNI 
UNITED STATES 


Level of Commitment (0.75) 


-Congress 3 
-OSD 4 
-Service 6 
-Strength Rating B25 


Programmatic Factors (1.5 

-Program Objective 

-Military Reqm’ts 

-MOU/Program 
Schedules 

-Workshare 

=~Eayort 

-Strength Rating 


Abi la ty “torContripute “Gao) 


O-~] 


iS 
> 
i230 


FOREIGN 


Level of Commitment (0.75) 


-Legislature 2 
-MOD is. 
-Service/Equiv. 6 
-Strength Rating a. 25 


Programmatic Factors (1.5) 


-Program Objective 
-Military Reqm’ts 
-MOU/Program 
Schedules 
-Workshare 5 
~Paviort 5 
-Strength Rating 8 


CO @O W 


Pel 


Ability to Contribute (Gila 


-Funding 5 -Funding 10 

-Government dig -Government aC 
Technology Technology 

-Strength Rating TsO -Strength Rating 10300 

Technolo Sees iie 1.0 Technolo Securit a0 

-Technology 10 -Technology 116 
Treanster Transier 

-Security 5 -Security 0 

-Strength Rating 1250 -Strength Rating 1030 


Zo 


OVERALL STRENGTH MATCHING 1. 


COMPOSITE STRENGTH 
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62.38 


PROGRAM SHORT TITLE: C2P2 


UNITED STATES FOREIGN 
Level of Commitment (0.75 Level of Commitment (0.75) 
-Congress a -Legislature 1 
-OSD Z - MOD 4 
-Service 4 -Service/Equiv. 6 
-Strength Rating L295 -Strength Rating Piedad brs. 
Programmatic Factors (1.5 Programmatic Factors (1.5) 
-Program Objective 1 -Program Objective 1 
-Military Reqm’ts al -Military Reqm’ts 1 
-MOU/Program 6 -MOU/Program 6 
Schedules Schedules 
-Workshare 10 -Workshare s20, 
-Payoft 6 -Payofft 6 
-Strength Rating Peaeio -Strength Rating Ee 
Boneerety to Contribute (1.0) Bad ieee Omeon te 7 Dutton 0) 
-Funding 2 -Funding 8 
-Government LO -Government LO 
Technology Technology 
-Strength Rating 6.00 -Strength Rating 9.00 
Technolo Securit a6 10) Technolo Securit 8 
-Technology 8 -Technology 8 
Transfer Transfer 
-Security 5 -Security 0 
=Strength Rating 650 -Strength Rating Suet) 


OVERALL STRENGTH MATCHING 1.25 


COMPOSITE STRENGTH 52.15 


oie 


ad. Results of Application of Revised Criteria and 

Method 
The results of the evaluation conducted using the revised 
criteria and method are summarized in the table below, 
compared, when applicable, to the results of the initial 


evaluation: 





D. ANOMALIES RESULTING FROM APPLICATION OF REVISED CRITERIA 
AND METHOD 
Several anomalies are readily apparent regarding the two 
evaluation procedures. Of primary significance is the fact 
that the four programs being evaluated for the first time 
generally scored lower than those programs being re-evaluated 


uSing the revised procedures. Additionally, for two pairs 
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(ASMCM/SPIN and CIP/CSNI), the revised rankings are the 


reverse of the initial rankings. These anomalies caused the 


IP office to question whether there might be problems with the 


revised procedure; evaluations using the revised procedure 


were hNalted until a review could be conducted. 
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IV: ANALYSIS OF DATA 


A. SPREADSHEET ANALYSIS OF INITIAL EVALUATION CRITERIA AND 
METHOD 
1. Construction of Spreadsheet 
To analyze the IPO evaluation process required translating 
the data from the various sets of evaluation sheets into a 
spreadsheet program that replicated both the appearance and 
the functions of the evaluation sheet. The cell contents of 
an example of this spreadsheet program are given below: 
Al: NATO MECHANICAL MINESWEEPING SYSTEM 
Hoe FOREIGN: FRANCE, SPAIN, 
A4: UNITED STATES 
E4: THE NETHERLANDS, ITALY 
A6: LEVEL OF COMMITMENT 


E6: LEVEL OF COMMITMENT 
A8: CONGRESS 


Ce a 
E8: LEGISLATURE 
G8: uf 
A9: OSD 
GS oar 
E9: MOD 
G9: 6 
Al0O: SERVICE 
CAO; 8 
E10: SERVICE/EQUIV. 
G10: 9 
All: WEIGHT 
Bie 0.475 
Billi: WEIGHT 
Bl idee O27 5 


Al2: STRENGTH RATING 

C12: @SUM{C€e.. C10) / eCcOUNIVes Clo) cae 
E1l2: STRENGTH RATING 

G12: @SUM(G8..G10) /@COUNT(G8..G10) *F11 
Al4: PROGRAMMATIC FACTORS 
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B14: 
Al6: 
CneG:: 
BLS 2 
G16: 
pl]: 
Cay: 
Bl]: 
G7: 
Als: 
anes:: 
ei 
G18: 
Al9: 
Gio: 
Eel: 
G18: 
A20: 
e220: 
20s 
G20: 
A21: 
C21: 
ja) 
G21 
EZ : 
p22: 
Bao: 
B22: 
AZ: 
E23): 
Bs: 
G23: 
A25: 
Be Si: 
A27: 
G27. 
|, 
G27: 
A28: 
e725. 
jg 23 2 
G28: 
A29: 
C29: 
BZ2o:: 
G29: 
A30: 
eo0": 
Be 
G20: 


PROGRAMMATIC FACTORS 
PROGRAM OBJECTIVES 
tia OBJECTIVES 
eany RQOMTS 
ey RQMTS 
Bc ROMTS 
ies RQOMTS 
ae SKED 
Ny eee SKED 
—— BASE IMPACT 
——— BASE IMPACT 
= PARTY SALES 
—_— PARTY SALES 
ee 


1.5 

STRENGTH RATING 

@SUM(C16..C21) /@COUNT(C16..C21) *B22 
STRENGTH RATING 

@SUM (G16. .G21) /@COUNT (G16..G21)*F22 
ABILITY TO CONTRIBUTE 

ABILITY TO CONTRIBUTE 

FUNDING 


GOVT TECHNOLOGY 
10 

GOVT TECHNOLOGY 
158 

INDUST TECHNOLOGY 
8 

ENE Sec HNOLOGY 
8 

TECH TRANSFER 

me 

TECH TRANSFER 

LO 
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A31: SECURITY 

C3: 

E31: SECURITY 

G31: 5 

A32: WEIGHT 

Ba2- ali 

E32: WEIGHT 

F322: di 

A33: STRENGTH RATING 

C33: @SUM(C27..C31) /@COUNT (C27. .C31) *B32 
E33: STRENGTH RATING 

G33: @SUM(G27..Geal) /@GOUNT (G27. G3 ie haz 
A35: TOTAL STRENGTH 

Cas: +C124+C23+C33 

E35: TOTAL STRENGTH 

Go: +G612+6234+G633 

D38: DEGREE OF CONVERGENCE 
D40: COMBINED 

H40: COMPOSITE 

D41: STRENGTH 

F41: WEIGHT 

H41: WEIGHT 

A43: LEVEL OF COMMITMENT 
D43: @VALUE(C12) +@VALUE (G12) 
F43: 1 

H43: +D43*F43 

A44: PROGRAMMATIC FACTORS 
D44: @VALUE (C23) +@VALUE (G23) 
Baa Gig 5 

H44: +D44*F44 

A45: ABILITY TO CONTRIBUTE 
D45: @VALUE (C33) +@VALUE (G33) 
F45: 1.5 

H45: +D45*F45 

E47: TOTAL COMPOSITE STRENGTH 
H47: +H43+H44+H45 


While creating the spreadsheet, it was discovered that, 


for any line item on the evaluation sheet not given a 


numerical score (i.e. "Security" under the United States’ 


"Ability to Contribute"), the corresponding cell gen the 


spreadsheet (C31) had to be completely blank. Any entry (1. 
e. NA, 0, or even a global justification command) in such a 


cell would be counted as an entry in the subsequent "@COUNT" 
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formula used to compute the area strength rating (in this 
instance cell C33) and would result in an incorrect 
eabeulation. 

2. Results of Spreadsheet Analysis 

The key formula cells of this spreadsheet (C12, G12, C23, 
G23, C33, G33, C35, G35, and H47) duplicated the values of the 
area strength ratings, the total strength ratings, and the 
composite strength rating of the evaluation sheet. The values 
represented by the other formula cells (D43, H43, D44, H44, 
D45, H45) do not appear on the evaluation sheet but were 
included in the spreadsheet for ease of computation. 

This spreadsheet analysis was applied to the evaluation 
sheets representing the initial evaluation performed on the 
following candidate programs: 

@® NATO Mechanical Minesweeping System 

® Anti-Ship Missile Countermeasure 

® "Spinnaker" 

@® U. S. PACOM Combined Interoperability Program 

® NATO CSNI 

® Continuous Processing of Solid Propellants 

@® Remote Sensing Unit with Synthetic Aperture Radar 
@® Airborne Electromagnetic Sensor 

® Tactical Lightweight Surveillance Satellite 

® Tactical Target Support 


® Distributed Tactical Surveillance 
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® Tactical Ship Surveillance 

® Oceanography Cooperation--Japan 

® Oceanography Cooperation- -Norway 

® Ocean Color Assessment (G-I-N Sea) 

® Ocean Color Assessment (Mediterranean Sea) 
® Millimeter Wave Sea Surveillance 


@® Remote Spectrometry Research. 


In every instance, the spreadsheet analysis of these programs 


directly correlated to all values on the evaluation sheets. 


B. SPREADSHEET ANALYSIS OF REVISED EVALUATION CRITERIA AND 
METHOD 
1. Construction of Spreadsheet 
A second, Similar spreadsheet based on the evaluation 
. Sheets of the revised evaluation criteria and procedures was 
then created; the criteria scores and weights of the Maritime 
Unmanned Aerial Vehicle (MUAV) evaluation sheet were used in 
creating the spreadsheet. The following represents the cell 
contents of this second spreadsheet: 
Al: MUAV 
EZ: FOREIGN, CANADA FRANCS 
E3: GERMANY, NETHERLANDS 
A4: UNITED STATES 
E4: NORWAY, UNITED KINGDOM 
A6é: LEVEL OF COMMITMENT 
EF6: LEVEL OF COMMITMENT 
A8: CONGRESS 


Ce sh 
E8: LEGISLATURE 
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SERVICE 

6 

SERVICE/EQUIV. 

6 

WEIGHT 

0.75 

WEIGHT 

0.75 

STRENGTH RATING 
@SUM(C8..C10) /@COUNT (C8. .C10) *B11 
STRENGTH RATING 
@SUM(G8..G10) /@COUNT (G8. .G10) *F11 
PROGRAMMATIC FACTORS 
PROGRAMMATIC FACTORS 
PROGRAM OBJECTIVES 

8 

PROGRAM OBJECTIVES 

8 

MILITARY ROMTS 

8 

MILITARY ROQMTS 

7i 

MOU/PROGRAM SKED 

10 

MOU/PROGRAM SKED 

10 

WORKSHARE 

5 

WORKSHARE 

5 

PAYOFF 

8 

PAYOFF 

8 

WEIGHT 


cee 

STRENGTH RATING 
@SUM(C16..C20) /@COUNT (C16. .C20) *B21 
STRENGTH RATING 

@SUM (G16. .G20) /@COUNT (G16. .G20) *F21 
Ap wigs LO CONTRIBUTE 

ABILITY TO CONTRIBUTE 

FUNDING 

10 


sik 


EZ 6) .0F UND ENG 


GZG6- 5 

PING Ts GOVT TECHNOLOGY 
C27 

Ba GOVT TECHNOLOGY 
G27 

AZ Ot WEIGHT 

B26: aL 

EZoe WEIGHT 

EZe. al 


A29: STRENGTH RATING 

C292 | @SUMI(C26. C27), GCOChT (2G ey ieaa ze 
B29: STRENGTH RATING 

G29: @SUM(G26..G27) /@COUNT (G26. .G27) *F28 
A31:  TECHNOLOGY/SECURITY 

B31: TECHNOLOGY/SECURITY 

A33: TECH TRANSFER 


555 186 

B33: TECH GEANSE EE 
C33: EG 

A34: SE CURL ITY 
Ga4e 10 

Dye le SECuRa iy 
Gan 

yo Ys WEIGHT 
Bao. al 

Eo WEIGHT 
Reo 1 


A36: STRENGTH RATING 

C362) @SUM(C332 7634) 7 GCCOUNT (C3326 lanes 
E36: STRENGTH RATING 

G36: _. @SUM (G33 5.2634) /@COUNT (633- G34) *Fe5 
A40: TOTAL COMBINED STRENGTH 

D40: +4+C124+G124+C224+6224+C294+G29+C36+G36 
A4l1: STRENGTH MATCHING 

bai: 1 

A43: TOTAL COMPOSITE STRENGTH 

D43-° 4D40-paI 


2. Results of Spreadsheet Analysis 

With this spreadsheet, the formula cells corresponding to 
the area strength ratings (C12, G12, C22, G22, C29, G2Z97eeec 
and G36) produced values identical with the values on the 


evaluation sheet. The composite strength formula cell (D43), 
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however, produced a value different from the evaluation sheet 
Cree vaee, 75 .20)m This second spreadsheet was applied to 
all the programs that had undergone the revised evaluation 
procedure, using the criteria scores and weights from the 
evaluation sheets. Always, the results were the same: area 
strength ratings corresponded to the evaluation sheets but 
composite strength ratings differed. The following table 


summarizes the results of this step in the analysis: 


- __ 


PROGRAM Syl gl gl chy De no ai gs sly 24 EVALUA- EVALUA- 
SHORT SHEET Shri TION TLoOnW 
SU We Be COMPOSITE | RANKING SHEET 


STRENGTH 
RATING STRENGTH 
RATING 
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Since the spreadsheet calculations corresponded to the 
evaluation sheets in all area strength ratings, and differed 
in all composite strength ratings, the cell governing the 


computation of "Composite Strength" was examined further. 


C. REASON FOR DISCREPANCY BETWEEN EVALUATION SHEETS AND 
SPREADSHEET ANALYSIS 
The "Composite Strength" value on the evaluation sheet was 
being calculated ina manner different from the intent of the 
evaluation sheet. Referring back to the MUAV spreadsheet 
analysis, "Composite Strength" is calculated by the formula 


"+4D40*D41" or, Substituting in for cell D40, by the formula 
"+ (C12+G12+C22+G22+C29+G29+C36+G36) *D41." 


This formula captures the intent of the revised procedure: to 
apply a Single "Strength Matching" weight (D41) to the sum of 
all the area strength ratings. 

The IPO evaluation sheets, however, calculate "Composite 


Strength" using the formula 
"4C124+G124+C224+G22+C36+G36+ ( (C29+G29) *D41) ." 


This formula isolates and sums the "Ability to Contribute" 
area strength ratings, applies the "Strength Matching" weight 
to that sum, and then adds in the remaining area strength 


ratings. By doing so, the formula is responsible for the 
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downward skewing noticed by the IP Office when they first 


applied the revised evaluation criteria. 


D. APPLYING THE SPREADSHEET CALCULATIONS TO THOSE CANDIDATE 
PROGRAMS NEVER EVALUATED WITH REVISED CRITERIA AND METHOD 
The spreadsheet program representing the intent of the 
revised criteria and procedure was then applied to the twelve 


programs never re-evaluated with the revised criteria. 


Criteria scores for the two line items ("Workshare" and 
"Payoff") not on the initial evaluation sheet had to be 
developed. For "Workshare," a value equivalent to the score 


given to "Industrial Base Impact" on the initial evaluation 
was used. For "Payoff," the subjectively determined "Payoff 
Potential" from the initial evaluation was translated into a 
numerical score between one and ten and used in the 
spreadsheet. The "Degree of Convergence" weight assigned to 
the "Ability to Contribute" area on the initial evaluation was 
used as the "Strength Matching" weight of the spreadsheet. 
The "Workshare" and "Strength Matching" assignments were done 
after consulting with Mr. Frank Kenlon of the IP Office; the 
"Payoff" assignment represents a natural progression from one 
format to another. The results of this analysis are shown in 


the table below: 


oie 


PROGRAM STRENGTH COMPOS GLE 
SHORT TATTLE WEIGHTING STRENGTH 
RATING 





E. FINAL PRIORITY RANKING OF ALL 22 CANDIDATE PROGRAMS 

The following table represents the final ranking of the 22 
programs based on the composite strength ratings calculated by 
the spreadsheet; included is a column with the current status 


of each candidate program: 


PROGRAM COMPOSITE CURRENT 
SHORT TITLE STRENGTH STATUS 


SIGNED MOU 


AWAITING START 
OF NEGOTIATIONS 
IN NEGOTIATION 


| PROPOSAL 
WITHDRAWN 
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PROGRAM COMP US we CURRENT 
SHOR] TITLE STRENGTH STATUS 
RATING 





CSNI 2.00 SIGNED MOU | 


AES SOG PROPOSAL 
WITHDRAWN 

ADV ST Cal aed S16) DEVELOPING RAN 

MUAV G2 DEVELOPING RAN 


CIP 62.10 AWAITING START 
OF NEGOTIATIONS 
fcap2, | 60.50 DEVELOPING RAN 
TLSS 58.63 PROPOSAL 
WITHDRAWN 
AL BAT 55.38 DEVELOPING RAN 
DTS 53.90 PROPOSAL 
WITHDRAWN 
TTS 53.25 PROPOSAL 
WITHDRAWN 


OCA-G oO PA PROPOSAL 
WITHDRAWN 


OCA-M S10, Sis: PROPOSAL 
WITHDRAWN 
Ou- uw 48.50 PROPOSAL 
WITHDRAWN 
OCW 48.50 PROPOSAL 
WITHDRAWN 
TSS 41.03 PROPOSAL 
WITHDRAWN 
RAIDS Ceara) PROPOSAL 
WITHDRAWN 


MWSS Eee PE | PROPOSAL 
WITHDRAWN 


RSR Soe 70 PROPOSAL 
WITHDRAWN 
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With the spreadsheet evaluation, it appears that a 
composite strength score of 55 or above represents a 
threshold; all programs that did not reach that threshold 
resulted in the candidate program being withdrawn from 
consideration. Two programs (RSU and AES) that exceeded the 
threshold also had their proposals withdrawn; to correlate 
their withdrawal to their evaluation scores, statistical 
analysis techniques were applied to the spreadsheet evaluation 


SCOLReGs- 


F. STATISTICAL ANALYSIS OF SPREADSHEET EVALUATION SCORES 

Based on the status of each of the 22 programs, the 
programs were assigned to one of two groups, Unsuccessful or 
Successful, to perform a statistical analysis of the 
spreadsheet evaluation scores. The Unsuccessful group 
consisted of those programs with a status indicating that the 
proposal had been withdrawn; the Successful group consisted of 
all others. 

Combined (U. S. and Foreign) median and sample mean scores 
for each of the four evaluation areas of the spreadsheet 
(Level of Commitment, Programmatic Factors, Ability to 
Contribute, and Technology/Security) were calculated for each 
group; the standard distribution of the mean was also 
calculated. The following four tables record the results of 


those calculations: 


Sb 


aa 


COMBINED LEVEL | UNSUCCESSFUL | SUCCESSFUL 
OF COMMITMENT | PROPOSALS PROPOSALS 
_ MEDIAN 4:63 


| SAMPLE MEAN | Bye Glie) osc 


STANDARD 0.33 2.00 
DEVIATION 


= 



















UNSUCCESSFUL | SUCCESSFUL 
j PROGRAMMATIC PROPOSALS PROPOSALS 








| FACTORS 

[MEDIAN 
SAMPLE MEAN | 815.44 | 19.50 
DEVIATION 


—— 


COMBINED UNSUGEHSSPUL WSUCCESSEUL 
ABIL L Gyo PROPOSALS PROPOSALS 
Cones suis 


MEDIAN ia 018 ie. O10 


SAMPLE MEAN VGr Ge 


STANDARD 
DEVIATION 


_ 
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Of all the median and sample means calculated, the most 


statistically significant, in terms of the range of score 


disparity between the Unsuccessful and Successful groups, are 


2g 


the sample mean scores of the Level of Commitment area. 
Combined median, sample means, and standard distributions were 
calculated for each line item in the Level of Commitment area; 


the results are shown in the three tables below: 
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Statistically, the most significant of these line item 





scores are the median and sample mean values of the combined 


OSD/MOD line item scores. Without OSD/MOD support, a proposal 


4,040, 


appears doomed; with OSD/MOD support, a proposal’s weaknesses 


in other areas may be surmounted. 


G. CORRELATION OF IPO’S EVALUATION CRITERIA WITH OSD SUCCESS 
INDICATORS 
At a Four Powers (United States, United Kingdom, France, 
Germany) Conference held in Monterey, California on October 5, 
1991, the Deputy Under Secretary of Defense for International 
Programs (DUSD(IP)) briefed the participants on OSD’s 
viewpoint regarding the criteria of success for cooperative 
programs; the OSD criteria are presented below: 
® A common, well defined requirement; 


@® An early start supported by technology cooperation and 
venture capital from Nunn amendment funding; 


® A clear view ahead on life-cycle funding; 
® A clear view of competitive programs; 

® Equal priority for each player; 

® Two or three partners maximum; 


@® Fach partner bringing something to the program besides 
money; 


Seine willl j€6 See it through, including a mutual 
understanding of cultures, concurrence on acquisition 


Strategy, strong program managers, and support at the 
highest level. 


The policy, procedures and criteria implemented by IPO 


through the evaluation of candidate programs generally 


Supports the OSD success criteria. One instance of divergence 


Oi. 


pertains to the number of partners being considered for a 
program; IPO does not take into account a candidate program 
having more than three partners while conducting its 
evaluation. Unless it has significant OSD/MOD support, a 
program with an excesSive number of partners may experience 
difficulties obtaining the authority to negotiate wean 


international agreement. 


OZ 


A. 


Vv. CONCLUSIONS AND RECOMMENDATIONS 


CONCLUSIONS 


The examination of the evaluation performed by the Navy 


International Programs Office on 22 candidate program 


proposals led to the following conclusions: 


® Upon revising the success criteria, a miscommunication 


resulted in evaluations being conducted in such a manner 
that overemphasized the "Ability to Contribute" area of 
the criteria and that produced inaccurate "Composite 
Strength" scores and ranking. 


The single, evaluation criteria line item that principally 
determines whether a candidate program gains approval to 
develop the Request Authority to Negotiate document is the 
level of OSD/MOD support of the program; by extension, the 
Single principal evaluation area is the "Level of 
Commitment" area. 


® The International Programs Office’s policy, procedures, 


and evaluation criteria are in general agreement with 
their OSD counterparts; however, the International 
Programs Office does not penalize a candidate program for 
having an excessive number of partners. 


RECOMMENDATIONS 


The recommendations derived from the above conclusions are 


as follows: 


@ The International Programs Office should correct the 


software programming error that produces the evaluation 
anomalies; this should be accomplished by implementing the 
spreadsheet "Composite Strength" formula in Section IV-C 
of this thesis. 


The International Programs Office should direct its 
resources at those candidate programs with a composite 


ce 3 


strength score of 55 or above, and at all candidate 
programs with a combined OSD/MOD line item score of three 
or above. 


The International Programs Office should subtract points 
from the composite strength scores of those candidate 
programs with an excessive number of partners before 
ranking to reflect the greater risk of the program. 


The International Programs Office should continue to 
Critique the criteria and procedures used to evaluate 
candidate programs to stay current with the rapidly 
changing environment of the international arena. 
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